Apparatus for improving image vibration suppression

ABSTRACT

An image sensing apparatus having a function of correcting vibration of an image limits operation of a vibration correction unit in correspondence with the ratio of the correction amount as an output from the vibration correction unit to the maximum correction range of that unit.

BACKGROUND OF THE INVENTION

The present invention relates to an apparatus for preventing orsuppressing vibrations of an image due to hand shakes and, moreparticularly, to an image sensing apparatus such as a video camera orthe like, which comprises an anti-vibration function.

Recent video cameras normally have an anti-shake function. Theanti-shake function includes optical correction and electroniccorrection.

In optical shake correction, a prism or lens member that can displacethe optical axis of image sensing light is inserted into the opticalpath of the image sensing light that becomes incident on an imagesensing element, and the optical axis is deflected in correspondencewith shake, thus canceling a motion of an image arising from shake. As ashake detection means, it is a common practice to directly detectvibration components acting on a camera using an angular velocity sensorsuch as a vibration gyro and to integrate the outputs from the sensor soas to detect angular displacement of the camera.

On the other hand, the electronic shake correction function is oftenused together with motion vector detection that calculates the movingamount of a camera on the basis of a change in video signal betweenadjacent fields. When a partial video signal is extracted and read outfrom an image stored in a field memory for motion vector detection, thevideo signal is extracted to remove the detected motion of the image. Asanother electronic shake correction, an angular velocity sensor is usedin vibration detection, and a partial image is extracted from an imageoutput from the image sensing element to cancel a motion of that imageand to output the extracted image.

Electronic shake correction is done at field periods since it isimplemented by an electronic process for a video signal. Hence, theelectronic shake correction cannot remove shake during exposure, but canattain a size and weight reductions compared to optical correction.Also, when a high-density, large image sensing element is used, theresolution of an image extracted from the sensed image signal can beincreased, and deterioration of image quality which is inferior to theoptical correction can be improved to some extent.

With a video camera, the user often senses an image while making acamera work such as panning, tilting, or the like, i.e., while he or sheis intentionally moving the camera. Upon image sensing with such camerawork, if the shake correction function is enabled, the limit of thecorrection range is reached soon to disturb the sensed image, and aresponse in a direction desired by the photographer is delayed. As ameans for preventing such problems, an arrangement that suppressescorrection capability by limiting shake correction is proposed (e.g.,Japanese Patent Laid-Open No. 5-142614). Also, an invention thatrealizes natural panning even when the focal length enters an ultratelescopic range is proposed (Japanese Patent Laid-Open No. 9-51466).

In conventional panning control, when the level of a shake detectionsignal (or its processed signal) has exceeded a predetermined thresholdvalue, the vibration correction capability is suppressed. Hence, incontinuous video sensing, i.e., motion image sensing, the panningcontrol makes the motion of an image on the frame during a transition toa panning mode unnatural.

In order to avoid such problems, an invention that changes the vibrationcorrection characteristics to smoothly change the vibration correctioncapability along with an elapse of time is also proposed (JapanesePatent Laid-Open No. 8-313950).

However, where the proposed technique is applied to an optical orelectronic anti-vibration device-in which a vibration correction meansis placed behind a zoom lens, it requires setting parameters for panningoperation for each change in focal length, hence such device cannot berealized in practice.

Further, the proposed technique has not given a full consideration tovarious image sensing conditions in how applying restraints to theanti-vibration correction.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an image vibrationprevention apparatus or image sensing apparatus that, in response to apanning operation, can easily and appropriately limit an correctingoperation of the apparatus.

The above object is achieved by providing an image vibration preventionapparatus or image sensing apparatus comprising:

a vibration correction device for correcting a vibration of an image;and

a limiting device for calculating a ratio of an amount of vibrationcorrection to be effected by said vibration correction device inaccordance with an image shake condition, to a maximum value ofvibration correction by said vibration correction device, and limitingan operation of said vibration correction device in correspondence witha calculation result.

The above object is also achieved by an image vibration preventionapparatus or image sensing apparatus comprising:

a vibration correction device for correcting a vibration of an image;and

a limiting device for, in response to a panning operation, limiting acorrection operation of said vibration correction device, by applyingdifferent limitations in pitch and yaw directions to said vibrationcorrection device.

Other features and advantages of the present invention will be apparentfrom the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram showing an arrangement according tothe first embodiment of the present invention;

FIGS. 2A and 2B are views showing the relationship between the entireimage sensing area and extraction range;

FIG. 3 is a flow chart showing the process for calculating angulardisplacement from angular velocity signals detected by angular velocitysensors 18 a and 18 b;

FIG. 4 is a flow chart showing the process for determining theextraction range from the angular displacement calculated by the processshown in FIG. 3;

FIG. 5 is a graph showing the cut-off frequency characteristics as afunction of the vibration correction amount at the most telescopicposition;

FIG. 6A is a graph showing the cut-off frequency characteristics as afunction of the vibration correction amount at the most wide-scopicposition;

FIG. 6B is a view for explaining the relationship between the focallength and camera client;

FIG. 7 is a schematic diagram showing an arrangement according to thesecond embodiment of the present invention;

FIG. 8 is a flow chart showing anti-vibration control of the secondembodiment show in FIG. 7;

FIG. 9 is a graph showing the characteristics of the limiting threshold(cutoff frequency) as a function of the normalized vibration correctionamount;

FIGS. 10A and 10B show examples of changes in effective image apertureand maximum correction range (maximum shift limit) as a function of thefocal length;

FIG. 11 is a flow chart that uses determination of the limitingthreshold based on the normalized vibration correction amount in thefirst embodiment shown in FIG. 1;

FIG. 12 is a graph showing the characteristics of the limiting threshold(cutoff frequency) as a function of the normalized vibration correctionamount used in step S35 in FIG. 11;

FIG. 13 is a schematic block diagram showing an arrangement according tothe third embodiment of the present invention;

FIG. 14 is a flow chart of the initial setup routine of ananti-vibration control module 254;

FIG. 15 is a flow chart of a routine for calculating the vibrationangle;

FIG. 16 is a flow chart of a process for determining the extractionrange from the angular displacement;

FIG. 17 is a block diagram showing the arrangement of an image sensingapparatus according to the fourth embodiment of the present invention;

FIGS. 18A and 18B are explanatory views showing an extraction frame ofthe fourth embodiment;

FIG. 19 is a flow chart showing anti-vibration control of the fourthembodiment;

FIG. 20 is a flow chart showing anti-vibration control of the fourthembodiment;

FIG. 21 is a graph showing the limiting amount characteristics of thefourth embodiment;

FIGS. 22A to 22C are graphs showing the limiting strength changecharacteristics of the fourth embodiment;

FIG. 23 is a block diagram showing the arrangement of an image sensingapparatus according to the fifth embodiment of the present invention;

FIG. 24 is a graph showing the limiting amount characteristics of thefifth embodiment;

FIG. 25 is a block diagram showing the arrangement of an image sensingapparatus according to the sixth embodiment of the present invention;

FIG. 26 is a flow chart showing anti-vibration control of the sixthembodiment;

FIG. 27 is a flow chart showing anti-vibration control of the sixthembodiment;

FIGS. 28A to 28C are explanatory views showing the limiting disabletiming of the sixth embodiment;

FIGS. 29A and 29B are explanatory views showing determination of thepanning period in the sixth embodiment;

FIG. 30 is a block diagram showing the arrangement of an image sensingapparatus according to the seventh embodiment of the present invention;

FIG. 31 is a flow chart showing the anti-vibration control sequence ofthe image sensing apparatus of the seventh embodiment;

FIG. 32 is a flow chart showing the anti-vibration control sequence ofthe image sensing apparatus of the seventh embodiment;

FIG. 33 is a flow chart showing the anti-vibration control sequence ofthe image sensing apparatus of the seventh embodiment;

FIG. 34 is a graph showing the relationship between the correctionamount and lookup address in the image sensing apparatus of the seventhembodiment;

FIG. 35 is a graph showing the relationship between the lookup addressand cutoff frequency in the image sensing apparatus of the seventhembodiment;

FIG. 36 shows an example of a data table in a data table memory in theimage sensing apparatus of the seventh embodiment;

FIG. 37 is a block diagram showing the arrangement of a band-passprocessor in the image sensing apparatus of the seventh embodiment;

FIG. 38 is a graph showing the change characteristics of the cutofffrequency at the beginning of panning (image sensing) in the imagesensing apparatus of the seventh embodiment;

FIG. 39 is a graph showing the change characteristics of the cutofffrequency at the end of panning (image sensing) in the image sensingapparatus of the seventh embodiment;

FIG. 40 is a block diagram showing an arrangement according to theeighth embodiment of the present invention;

FIG. 41 is a flow chart for explaining vibration correction;

FIG. 42 is a flow chart for explaining vibration state discrimination;

FIGS. 43A and 43B are graphs showing the vibration correctioncharacteristics;

FIGS. 44A to 44C are graphs showing a plurality of panningcharacteristics in correspondence with image sensing situations; and

FIG. 45 is a block diagram showing the arrangement of a modification(second modification) of the eighth embodiment.

DETAILED DESCRIPTION OF THE INVENTION

The embodiments (first to eighth embodiments) of the present inventionwill be described in detail hereinafter with reference to theaccompanying drawings.

The first and third embodiments are premised on the electronicanti-vibration scheme, i.e., a scheme that implements anti-vibration bymoving the read position of image data read out from an image sensingelement and stored in a memory in accordance with the amount ofvibration, and the second embodiment is premised on the opticalanti-vibration scheme.

First Embodiment . . . FIGS. 1 to 7

FIG. 1 is a schematic block diagram showing the arrangement according tothe first embodiment of the present invention, in which a video camerahas an electronic anti-vibration function. Referring to FIG. 1,reference numeral 10 denotes a photographing lens; and 12, a CCD imagesensing element that converts an optical image formed by thephotographing lens 10 into an electrical signal. The output signal fromthe image sensing element 12 is amplified by an amplifier 14, and isinput to a camera signal processing circuit 16. The camera signalprocessing circuit 16 performs known camera signal processes such asgain adjustment, color balance adjustment, Y correction, and the likefor the image signal received from the amplifier 14, and forms andoutputs a standard video signal.

Reference numeral 18 a denotes an angular velocity sensor in a pitchdirection; 18 b, an angular velocity sensor in a yaw direction; 20 a and20 b, amplifiers for respectively amplifying the outputs from theangular velocity sensors 18 a and 18 b; and 22, an anti-vibrationcontrol circuit for detecting vibration of the camera main body and itsangle from the outputs (i.e., angular velocities in the pitch and yawdirections) from the amplifiers 20 a and 20 b, and canceling thevibration. More specifically, the anti-vibration control circuit 22comprises a microcomputer, which incorporates an A/D converter (notshown in FIG. 1) that converts the outputs from the amplifiers 20 a and20 b into digital signals. Reference numeral 24 denotes ananti-vibration ON/OFF switch that the user uses to instruct ananti-vibration function to the anti-vibration control circuit 22; 26, aCCD drive circuit for driving the image sensing element 12 in accordancewith a command from the anti-vibration control circuit 22 to read out adesired line portion; 28, a line memory for selecting an output imageportion in a line direction; and 30, a memory control circuit forcontrolling the line memory 28 in accordance with a command from theanti-vibration control circuit 22.

The anti-vibration control circuit 22 calculates angular displacementsby integrating the detected angular velocities (the outputs from theamplifiers 20 a and 20 b), calculates the pixel movement amount (nearlycorresponding to f×tanθ) on the image sensing element 12 resulting fromvibration on the basis of the obtained angular displacements, i.e.,vibration angles θ of the camera and a focal length f of thephotographing lens 10, changes the memory read start position to cancelthat pixel movement amount, and reads out an image signal from the imagesensing surface of the image sensing element or the corresponding areain the memory.

An image region extracted by electronic anti-vibration control will bebriefly explained below.

FIG. 2A shows the relationship between an image obtained from the entireimage sensing area of the image sensing element 12, and an image to beoutput extracted from that area. Reference numeral 32 denotes an entireimage sensing area; and 34, an extraction range of the entire imagesensing area 32 from which an image to be output is extracted. Thecoordinate position of the extraction range 34 is defined by its upperleft position coordinates (V₀, H₀). An image extracted from theextraction range 34 is edited and output to have an output image size orpixel format, as shown in FIG. 2B.

By moving the position of the extraction range 34 on the entire imagesensing area 32 to cancel vibration, the vibration can be corrected. Therange within which the position of the extraction range 34 can bechanged, i.e., vibration correction capability, is determined bydifferences of the numbers of horizontal/vertical pixels (to be referredto as the “numbers of extra pixels” hereinafter) between the entireimage sensing area 32 and extraction range 34. The range can berepresented by a dotted area 35 within which the upper left corner ofthe extraction range 34, for example, can move. The reference position(initial position) of the extraction range 34, denoted as (V_(c), H_(c))is determined in advance at a position when the vibration correction isOFF, specifically, at the position of the upper left corner of the range34 when it is positioned so that the center position of the range 34matches that of the entire range 32, more specifically at the positionhaving vertical and horizontal offsets with the half numbers of theextra pixels from the upper left corner of the entire range 32. In theexample of FIG. 2A, the reference position (V_(c), H_(c)) isoccasionally close to the corner position (V₀, H₀) of the range 34.V_(c) will be referred to as a vertical origin position hereinafter, andH_(c) will be referred to as a horizontal origin position hereinafter.

As methods of extracting an image contained in the extraction range 34,the first method of storing the entire image of the entire image sensingarea 32 in the field memory and interpolating only an image of theextraction range 34 in the horizontal and vertical directions to obtaina display size while reading out that image, and the second method ofusing a high-density, high-resolution type large-scale image sensingelement, the extraction range 34 of which satisfies the number ofscanning lines and the number of horizontal pixels required for astandard video signal, are available.

In the first embodiment, the image sensing element 12 uses a versatilePAL CCD image sensing element, and the output video signal complies withNTSC. The PAL CCD image sensing element has a high vertical pixeldensity. Using this fact, in the first embodiment, the verticalextraction position can be changed by changing the number of lines whichundergo fast discharging control within the range of extra linesexceeding the number of lines of NTSC, in correspondence with thevibration angle value. In the horizontal scanning direction, therelationship between the write start pixel position and the read startpixel position with respect to the line memory 28 is changed whileexecuting enlargement in correspondence with the selected aspect ratiousing the line memory 28 and memory control circuit 30, thus changingthe horizontal frame position.

In the first embodiment, the anti-vibration control circuit 22 makes theCCD drive circuit 26 execute fast discharging control in the verticaldirection, thereby reading out only a desired vertical scanning lineportion from the image sensing element 12. On the other hand, in thehorizontal scanning direction, a line image processed by the camerasignal processing circuit 16 is temporarily written in the line memory28, and enlargement (which can be implemented by decimating and readingout pixel data by changing the memory read rate) is simultaneously madein correspondence with the selected aspect ratio while changing the readposition of the line image in the line memory 28 in accordance with thepixel movement amount for vibration correction by the memory controlcircuit 30. With this control, read control for anti-vibration controlcan be implemented in the vertical and horizontal directions even uponusing a PAL CCD.

A signal read out from the line memory 28 undergoes a color process orthe like in the camera signal processing circuit 16, and is convertedinto a standard video signal to be output.

The anti-vibration control by the anti-vibration control circuit 22 willbe described in detail below with reference to FIGS. 3 and 4.

One objective of the first embodiment is to smoothly switchanti-vibration control in response to panning. In the first embodiment,the limiting amount is changed according to characteristics which changein proportion to the n-th power (n is an integer equal to or largerthan 1) of the vibration correction amount, thereby automaticallyadjusting the effect of vibration correction.

FIG. 3 is a flow chart of a process for calculating angulardisplacements by integrating angular velocity signals detected by theangular velocity sensors 18 a and 18 b. This process is an interruptprocess at a given period. In the first embodiment, this processing isexecuted at a frequency 10 times the field frequency, i.e., at 600 Hz incase of NTSC. This frequency corresponds to the sampling frequency ofangular velocity signals, i.e., the calculation frequency of angulardisplacements. An interrupt signal for this process can be generated bya known method. For example, clock signals are counted up or down, andupon counting clock signals corresponding to 1/600 sec, an interruptsignal is generated. The anti-vibration control circuit 22 samples theangular velocity signals by converting them into digital signals usingits internal A/D converter. In the first embodiment, assume that the A/Dconverter operates in a scan mode, and always converts an input signalinto a digital signal, for the sake of easy understanding.

The DC component is removed from the angular velocity signal sampled bythe A/D converter (step S1), and the band of the angular velocity signalis limited by cutting its AC component (step S2). The band limitation isthe same high-pass filter process as that for removing the DC component,except that its cutoff frequency is fixed in step S1 but is variable instep S2. By changing the cutoff frequency from the low- tohigh-frequency side, a desired band component is extracted. In the firstembodiment, the cutoff frequency is increased to lower the suppressioncapability of anti-vibration during camera work such as panning or thelike, and is decreased to obtain a sufficient vibration correctioneffect in normal image sensing. In order to also prevent an unnaturalimage from being formed when a correction limit is reached in correctingvibration larger than the upper limit of the vibration correction range,the cutoff frequency in step S2 is preferably adjusted. How to controlthe cutoff frequency will be explained later with reference to FIG. 4.

Angular displacement is calculated by integrating the band-limitedangular velocity signal (step S3). The calculated angular displacementcorresponds to the vibration angle acting on the camera main body. Avariable m indicating the number of times of vibration anglecalculations is incremented (step S4). If m=10 (step S5), 0 issubstituted in m (step S6) to end the interrupt process; if m≠10 (stepS5), the process ends directly. More specifically, if 10 interrupts havebeen generated per field period, m is reset to 0.

Steps S1 to S3 are executed for both the pitch and yaw directions.

A process shown in FIG. 4 is executed once per field. More specifically,the process shown in FIG. 4 is executed when m=0, i.e., immediatelyafter the end of the process shown in FIG. 3 for one field.

The control waits until m equals 0 (step S11). When the interruptprocess shown in FIG. 3 has been executed 10 times for the currentfield, m is reset to 0 (step S6). If m=0 (step S11), the vibrationcorrection amount is calculated (step S12). The vibration correctionamount is calculated by f×tanθ using the vibration angle θ and the focallength f of the optical system. A limiting threshold that limitsvibration correction capability is calculated from the calculatedvibration correction amount (step S13). The limiting threshold to becalculated is the cutoff frequency used in step S2 in FIG. 3, inpractice.

The target position coordinates (V₀, H₀) of the extraction range arecalculated (step S14). The target position coordinates are given by:$\begin{matrix}\begin{matrix}{V_{0} = \quad {V_{C} \pm {{number}\quad {of}\quad {pixels}\quad {to}\quad {be}\quad {moved}\quad {to}{\quad \quad}{correct}\quad {pitch}}}} \\{\quad {{vibration}\quad {angle}}} \\{= \quad {V_{C} \pm {\left( {- 1} \right) \times {focal}\quad {length} \times \tan \quad \frac{{pitch}\quad {vibration}\quad {angle}}{{vertical}\quad {cell}{\quad \quad}{size}}}}}\end{matrix} & (1) \\\begin{matrix}{H_{0} = \quad {H_{C} \pm {{number}{\quad \quad}{of}\quad {pixels}{\quad \quad}{to}\quad {be}\quad {moved}{\quad \quad}{to}\quad {correct}\quad {yaw}}}} \\{\quad {{vibration}{\quad \quad}{angle}}} \\{= \quad {H_{C} \pm {\left( {- 1} \right) \times {focal}{\quad \quad}{length} \times \tan \quad \frac{{yaw}\quad {vibration}{\quad \quad}{angle}}{{horizontal}\quad {cell}\quad {size}}}}}\end{matrix} & (2)\end{matrix}$

where $\begin{matrix}{{{vibration}{\quad \quad}{correction}\quad {amount}} = {{focal}\quad {length} \times \tan \quad \frac{{vibration}\quad {angle}}{{vertical}\quad {cell}\quad {size}}}} & (3)\end{matrix}$

With these equations, the numbers of pixels to be moved required forvibration correction are obtained.

A predetermined command, which includes the calculated target positioncoordinates (V₀, H₀) as the reference position coordinates of theextraction range, is output to the CCD drive circuit 26 and memorycontrol circuit 30 (step S15). The CCD drive circuit 26 and memorycontrol circuit 30 operate to execute extraction according to thecommand in the next field. After that, the flow returns to step S11 toprepare for the next field, and the control waits until integrationrepeats itself 10 times.

FIGS. 5 and 6A show the cutoff frequency characteristics as a functionof the vibration correction amount. FIG. 5 shows the characteristicswhen the lens is located at the most telescopic position, and FIG. 6Ashows the characteristics when the lens is located at the mostwide-scopic position. In both FIGS. 5 and 6A, the abscissa plots thevibration correction amount (=f·tanθ), and the ordinate plots the cutofffrequency f_(c). Since the cutoff frequency is determined as a functionof the vibration correction amount, the suppression capability ofanti-vibration can be finely controlled in correspondence with therequired vibration correction level, and can be smoothly switched evenupon panning. In the first embodiment, the maximum cutoff frequency isset at 6 Hz. This is because principal vibration frequency componentsfall within the range below 5 Hz.

The reason why the cutoff frequency is changed in proportion to thesquare of the correction amount in a region where the vibrationcorrection amount is small (a region where the correction amount <c₀ inFIG. 5) is to sharply raise the cutoff frequency compared to a casewherein the cutoff frequency linearly changes in proportion to thecorrection amount, when the vibration correction amount increases, andto set a low cutoff frequency as much as possible so as to improve theanti-vibration effect, when the vibration correction amount is in theneighborhood of zero.

When the range with a high anti-vibration effect (a range where thevibration correction amount is around zero) is to be maximized, theorder of the vibration correction amount is raised (e.g., the order israised to the third power, fourth power, and so on), so thatcoefficients and the like are set to sharply raise the cutoff frequencyas the vibration correction amount becomes larger.

In a telescopic region with a large focal length, the displacementvelocity of an object tends to be higher than that in a wide-scopicregion if the vibration correction amount remains the same. This isbecause, as shown in FIG. 6B, a vibration angular displacement (θ, θ′)corresponding to a given vibration correction amount has a smallervibration angular displacement θ_(T) in a telescopic region with a largefocal length (f_(T)) than a vibration angular displacement θ_(W) in awide-scopic region with a small focal length (f_(W)) and, hence, thevibration angle readily becomes large even at a low panning velocity.Hence, in order to prevent the correction amount from reaching avibration correction limit and the sensed image from being disturbed,the cutoff frequency must be raised more quickly at the telescopic sideas can be seen from a comparison between FIGS. 5 (telescopic side) and6A (wide-scopic side).

In this manner, by changing the cutoff frequency characteristics incorrespondence with the focal length, both anti-vibration control forpanning and “collision prevention” against a correction limit can beachieved using identical limiting parameters.

With the process described above with reference to FIGS. 3 to 6A, in thefirst embodiment, since the limiting amount for the vibration correctionamount, i.e., the cutoff frequency is changed in correspondence with thevibration correction amount, the limiting amount on the vibrationcorrection amount changes continuously, and anti-vibration control fornormal image sensing can be smoothly switched to that for panning.Especially, by changing the cutoff frequency in proportion to the n-thpower (n is an integer equal to or larger than 1 and assumes severaldifferent values) of the vibration correction amount, when the vibrationcorrection amount has a predetermined value, the vibration correctionamount can be sharply limited, or limited as much as possible, or thecorrection amount range without any limitation can be broadened, thusallowing flexible setups. As can be seen from a comparison between FIGS.5 and 6A, by setting appropriate limiting characteristics incorrespondence with the focal length, prevention of disturbance of thesensed image upon collision against a vibration correction limit, andanti-vibration control for panning can be implemented using identicalparameters for band limitations.

Modifications of First Embodiment

M-1: In the first embodiment, the PAL CCD image sensing element and linememory are used. Alternately, the same effect can be obtained bycontrolling the image extraction position on a field memory. Also, alarge-scale or ultra-high-resolution CCD image sensing element that doesnot require any enlargement control may be used.

M-2: In the first embodiment, angular velocity sensors are used as thevibration detection means. Alternatively, acceleration sensors may beused. In such case, another integration process need only be addedinside or outside the anti-vibration control circuit 22. The vibrationangular displacement may be calculated by either software or hardware.

M-3: In the first embodiment, a band limiting means that limits the bandof a vibration signal is used as the vibration correction limitingmeans. However, the present invention is not limited to such specificmeans. For example, the correction amount may be limited by changing thevibration correction gain. In this case, a motion vector detection meanscan be used as the vibration detection means.

Second Embodiment

The second embodiment in which an anti-vibration mechanism of thepresent invention is applied to an image sensing apparatus that uses azoom lens or an exchangeable lens type image sensing apparatus, will bedescribed below.

FIG. 7 is a block diagram of the second embodiment. The secondembodiment uses an optical vibration correction device that correctsvibration by moving a vibration correction shift lens in a directionperpendicular to the optical axis. The second embodiment comprises anoptical anti-vibration device in which a vibration correction means thatis vulnerable to the first problem mentioned above is placed behind azoom lens, but can clear this problem.

Reference numeral 110 denotes an inner-focus photographing lens, whichis composed of a stationary lens 112, zoom lens 114, stop 116,anti-vibration shift lens 118, and focus lens 120.

Reference numeral 122 denotes an image sensing element for converting anoptical image formed by the photographing lens 110 into an electricalsignal; 124, an amplifier for amplifying an output from the imagesensing element 122; and 126, a camera signal processing circuit forperforming a known camera signal process for the signal output from theamplifier 124.

A motor drive circuit 128 moves the zoom lens 114 in the optical axisdirection using a stepping motor 130. A motor drive circuit 132 movesthe anti-vibration shift lens 118 in a direction perpendicular to theoptical axis using a stepping motor 134. An encoder 136 detects theposition of the anti-vibration shift lens 118. An amplifier 138amplifies the output from the encoder 136, and a subtractor 140subtracts the output from the amplifier 138 from a control signalsupplied from a system control circuit 146 and supplies the differenceto the motor drive circuit 132. A motor drive circuit 142 moves thefocus lens 120 in the optical axis direction using a stepping motor 144.

The system control circuit 146 is a microcomputer that controls theoverall apparatus, controls the motor drive circuits 128 and 142, andoutputs a target value for anti-vibration control to the subtractor 140.

Reference numeral 148 denotes a zoom key which is operated by the userto change focal length. The system control circuit 146 moves the zoomlens 114 in a designated direction via the motor drive circuit 128 and130 in accordance with operation of the zoom key 148.

Reference numeral 150 a denotes a pitch angular velocity sensor; and 150b, a yaw angular velocity sensor. The detection outputs from the angularvelocity sensors 150 a and 150 b are amplified by amplifiers 152 a and152 b and the amplified signals are supplied to the system controlcircuit 146. Reference numeral 154 denotes an anti-vibration controlON/OFF switch.

The system control circuit 146 converts analog outputs from theamplifiers 152 a and 152 b into digital signals, and integrates thedigital signals to convert them into angular displacements. The systemcontrol circuit 146 corrects vibration by moving the shift lens 118 in adirection perpendicular to the optical axis, so as to move the sensedimage on the image sensing element 122 that has moved by vibration (byan amount nearly corresponding to f×tanθ) in a direction opposite to themoving direction due to vibration, on the basis of the obtained angulardisplacements, i.e., vibration angles θ and the focal length f of thephotographing lens 110. More specifically, the system control circuit146 outputs a vibration correction target value to the subtractor 140.The subtractor 140 compares the output from the amplifier 138 (i.e., asignal indicating the position of the shift lens 118) with the targetvalue, and supplies their difference signal to the motor drive circuit132. In this manner, the shift lens 118 moves to a positioncorresponding to the target value.

Also, the system control circuit 146 controls the zoom lens 114 andfocus lens 120. That is, the system control circuit 146 controls themotor drive circuit 128 in accordance with a signal coming from therotary zoom switch 148, the resistance of which changes incorrespondence with the pressure exerted, so as to move the zoom lens114 in the designated direction. The system control circuit 146 controlsthe motor drive circuit 142 to maximize a focus signal obtained from thecamera signal processing circuit 126 so as to move the focus lens 120along the optical axis, thus forming an optical image on the imagesensing surface of the image sensing element 122.

The anti-vibration control in the system control circuit 146 will beexplained below with reference to FIG. 8. In the second embodiment, avibration correction amount is normalized by the focal length andmaximum correction limit, and the limiting threshold is calculated tofollow predetermined limiting characteristics in accordance with thenormalized vibration correction amount. Hence, the second embodiment cancope with all focal lengths by only a single type of limitingcharacteristics.

In the second embodiment as well, angular displacements are calculatedby integrating the angular velocity signals detected by the angularvelocity sensors 150 a and 150 b so as to calculate the vibrationcorrection amount and its limiting threshold. In the first embodiment,the control period for vibration correction matches the field period.However, since the second embodiment uses an optical anti-vibrationsystem, the vibration sampling period can be matched with the vibrationcorrection period, and vibration can be corrected even during the chargeaccumulation time of the image sensing element. The process shown inFIG. 8 is an interrupt process executed at given periods by the systemcontrol circuit 146: e.g., at a frequency of 1 kHz in the secondembodiment. For example, signals obtained by frequency-dividingoscillation clocks at a predetermined frequency division ratio arecounted up (or down), and upon counting a time corresponding to 1 msec,an interrupt signal is generated. The system control circuit 146incorporates an A/D converter for converting the analog outputs from theamplifiers 152 a and 152 b into digital signals. Assume that the A/Dconverter always operates in a scan mode.

The system control circuit 146 removes the DC component from an angularvelocity signal sampled by the A/D converter (step S21), and limits theband of the AC component of the angular velocity signal (step S22). Theband limitation is the same high-pass filter process as that forremoving the DC component, except that its cutoff frequency is fixed instep S21 but is variable in step S22. By changing the cutoff frequencyfrom the low-to high-frequency side, a desired band component isextracted. In the second embodiment, as in the first embodiment, thecutoff frequency is increased to lower the suppression capability ofanti-vibration during camera work such as panning or the like, and isdecreased to obtain a sufficient vibration correction effect in normalimage sensing. In order to also prevent an unnatural image from beingformed when a correction limit is reached in correcting vibration largerthan the upper limit of the vibration correction range, the cutofffrequency in step S22 is adjusted.

Angular displacement is calculated by integrating the band-limitedangular velocity signal (step S23). The calculated angular displacementcorresponds to the vibration angle acting on the camera main body. Then,the vibration correction amount (shift target value) is calculated (stepS24). The vibration correction amount is given by f×tanθ using thevibration angle θ and focal length f of the optical system, as mentionedabove. The calculated vibration correction amount is normalized by aknown maximum correction limit (a movement limit of the shift lens 118)(step S25):

normalized pitch vibration correction amount=pitch vibration correctionamount/pitch maximum shift limit×100(%)  (4)

normalized yaw vibration correction amount=yaw vibration correctionamount/yaw maximum shift limit×100(% )  (5)

The limiting threshold that limits vibration correction capability iscalculated from the normalized vibration correction amount calculated inthis way (step S26). The limiting threshold to be calculated is thecutoff frequency used in step S22. The calculated cutoff frequency isused in the next band limitation. For example, when the cutoff frequencyassumes a large value, vibration components having frequencies equal toor lower than that cutoff frequency are cut off, thus weakening thevibration correction effect.

The correction amount (the target value of the shift lens 118)calculated in step S24 is output to the subtractor 140 (step S27).

FIG. 9 shows the characteristics of the limiting threshold (cutofffrequency) as a function of the normalized value of the vibrationcorrection amount. The abscissa plots the normalized vibrationcorrection amount, which assumes 100% when correction is done byshifting the lens to its maximum shift limit. The normalized vibrationcorrection amount indicates the ratio of a correction amount requiredfor correcting current vibration to the maximum correction amount. Theordinate plots the cutoff frequency serving as a limiting threshold. Asin the first embodiment, the cutoff frequency is a function of thesquare of the normalized vibration correction amount.

The maximum shift limit is determined as follows with reference to FIGS.10A and 10B. FIG. 10A shows a change in diameter of an effective imagecircle (to be referred to as an effective image aperture hereinafter) asa function of the focal length, and FIG. 10B shows a change in maximumcorrection range (maximum shift limit) as a function of the focallength. In FIG. 10A, the abscissa plots the focal length, and theordinate plots the effective image aperture. In FIG. 10B, the abscissaplots the focal length, and the ordinate plots the maximum correctionrange.

Referring to FIG. 10A, if A represents the minimum effective imageaperture when the shift lens 118 moves within the maximum mechanicalmovement range, characteristics 160 represent those of the photographinglens that can prevent any eclipse of the photographing screen at allfocal lengths from the wide-scopic side to the telescopic side even whenthe shift lens 118 mechanically moves to its maximum movement limit.Hence, the maximum value of the correction range obtained when a lensthat exhibits the characteristics 160 is used assumes a constant value Bshown in FIG. 10B. On the other hand, with a photographing lens that hascharacteristics 162 shown in FIG. 10A, an effective image aperturelarger than (or equal to) A is obtained only when the focal length isset on the telescopic side of C. When the lens with such characteristics162 is used at a focal length on the wide-scopic side of C, if the shiftlens 118 is shifted to its mechanically movable limit, the photographingscreen is partially eclipsed. Hence, the maximum correction range forthe lens with the characteristics 162 must be set at a value smallerthan B in a region on the wide-scopic side of the focal length C. Forexample, as indicated by characteristics 164 shown in FIG. 10B, as thefocal length moves to the wide-scopic side, the correction range islimited to a smaller value. In general, a lens optical system isdesigned to have the characteristics 162 to attain a size reduction ofthe lens.

In this fashion, even when the maximum correction range (the maximumvalue of the correction range) changes in correspondence with the focallength (e.g., like the characteristics 164), since the second embodimentnormalizes the vibration correction amount by the maximum correctionrange value, prevention of collision against the limit, and smooth shiftand release of panning can be realized without changing the limitingcharacteristics in a plurality of ways in units of focal lengths (i.e.,without providing a large number of characteristic change parameters).

Modification of Second Embodiment . . . First Modification

Determination of the limiting threshold based on the normalizedvibration correction amount can also be applied to the electronicvibration correction system in the first embodiment shown in FIG. 1.FIG. 11 shows the control sequence of a modification when determinationof the limiting threshold based on the normalized vibration correctionamount according to the second embodiment is used in the correctionsystem of the first embodiment shown in FIG. 1.

The control waits until m equals 0 (step S31). When the initializationprocess shown in FIG. 3 has been executed 10 times in the current field,m is reset (step S6). If m=0 (step S31), the vibration correction amountis calculated (step S32). The vibration correction amount is given byf×tanθ (=vibration correction amount) using the vibration angle θ andfocal length f of the optical system, as described above. As in stepS14, the target position coordinates (V₀, H₀) of the extraction rangeare calculated based on the obtained vibration correction amount (stepS33). In this way, the number of pixels to be moved required forvibration correction is obtained.

The vibration correction amount calculated in step S32 is normalized(step S34) by:

normalized pitch vibration correction amount=pitch vibration correctionamount/vertical pixel size/number of extra vertical pixels/2×100(%)  (6)

normalized yaw vibration correction amount=yaw vibration correctionamount/horizontal pixel size/number of extra horizontalpixels/2×100(%)  (7)

The first embodiment shown in FIG. 1 uses the PAL CCD image sensingelement 12 (582 vertical pixels×752 horizontal pixels: aspect ratio582/752≈0.78) in the NTSC camera. Upon extracting 485 vertical linescomplying with NTSC from the PAL image sensing element 12, if theabove-mentioned aspect ratio is to be maintained, the number ofhorizontal pixels to be extracted is 627. Hence, the number of extrapixels (of an ineffective image) is 97 vertical pixels×125 horizontalpixels. Since the sign of the correction direction changes incorrespondence with the direction of vibration, half extra pixels, i.e.,48.5 vertical pixels and 62.5 horizontal pixels, are used innormalization.

The limiting threshold (cutoff frequency) is calculated from theobtained normalized vibration correction amount (step S35). Anextraction command, which includes the target position coordinates (V₀,H₀) calculated in step S33 as reference position coordinates of theextraction range, is output to the CCD drive circuit 26 and memorycontrol circuit 30 (step S36). The flow returns to step S31 to preparefor the next field, and the control waits until integration repeatsitself 10 times.

FIG. 12 shows the characteristics of the limiting threshold value(cutoff frequency) as a function of the normalized vibration correctionamount, which is used in step S35 in FIG. 11. The abscissa plots thenormalized vibration correction amount, and the ordinate plots thecutoff frequency. Note that the abscissa defines 100% vibrationcorrection amount when correction is done using all pixels half theextra pixels as a maximum correction limit. In the first modification,the cutoff frequency is a function of the square of the normalizedvibration correction amount as in the second embodiment. To prevent thecorrection amount from reaching a vibration correction limit and thesensed image from being disturbed, in FIG. 12, as the vibrationcorrection amount comes closer to the maximum correction limit, thecutoff frequency is set to sharply shift to a larger value (higherfrequency).

In the first modification, since the vibration correction amountcalculated according to vibration is normalized by the maximum vibrationcorrection range, and the limiting threshold of a vibration signal isdetermined in correspondence with the normalized vibration correctionamount, even in a camera with a variable focal length or a camera with avariable effective image aperture corresponding to the focal length,smooth panning mode shift can be realized. Also, similar panningcharacteristics can be obtained by simple parameter setups irrespectiveof the anti-vibration scheme used.

Third Embodiment

The above embodiment is characterized by realizing a smooth panningprocess by simple parameter setups independently of the anti-vibrationscheme used. In the third embodiment to be described below, even whennot only the anti-vibration scheme but also the lens, image sensingelement, and video signal format vary, a panning process that canachieve the effect of the present invention can be done, and uniformsuppression capability of anti-vibration can be obtained.

FIG. 13 is a schematic block diagram of the third embodiment. In thethird embodiment, the present invention is applied to an exchangeablelens type video camera.

The third embodiment is composed of a lens unit 210 and camera main body240, and the lens unit 210 is detachable from the camera main body 240.

The lens unit 210 is of inner focus type, and comprises a stationarylens 212, zoom lens 214, stop 216, stationary lens 218, and focus lens220. A motor drive circuit 222 moves the zoom lens 214 in the opticalaxis direction using a stepping motor 224. A motor drive circuit 226moves the focus lens 220 in the optical axis direction using a steppingmotor 228. Reference numeral 230 denotes a camera control circuitcomprising a microcomputer, which controls the lens unit 210 viacommunications with the camera main body 240, and outputs information ofthe lens unit 210 to the camera main body 240. Reference numeral 232denotes a rotary zoom switch, the resistance of which changes incorrespondence with the pressure exerted.

The lens control circuit 230 controls the motor drive circuit 222 tomove the zoom lens 214 in the designated direction in accordance withoperation of the zoom switch 232. Also, the lens control circuit 230moves the focus lens 220 in the optical axis direction via the motordrive circuit 226 and motor 228 to maximize focus signal informationfrom the camera main body 240 on the basis of that information.

In the camera main body 240, reference numeral 242 denotes a CCD imagesensing element that converts an optical image formed by the lens unit210 into an electrical signal. The output signal from the image sensingelement 242 is amplified by an amplifier 244, and the amplified signalis input to a camera signal processing circuit 246. The camera signalprocessing circuit 246 performs known camera signal processes such asgain adjustment, color balance adjustment, γ correction, and the likefor the image signal received from the amplifier 244, and forms andoutputs a standard video signal.

Reference numeral 248 a denotes a pitch angular velocity sensor; 248 b,a yaw angular velocity sensor; and 250 a and 250 b, amplifiers forrespectively amplifying the outputs from the angular velocity sensors248 a and 248 b. Reference numeral 252 denotes a system control circuitcomprising a microcomputer, which communicates with the lens controlcircuit 230 of the lens unit 210, and controls the overall apparatus.The system control circuit 252 comprises an anti-vibration controlmodule 254, which detects vibration and its angle of the camera mainbody on the basis of the outputs (i.e., pitch and yaw angularvelocities) from the amplifiers 250 a and 250 b, and cancels vibration.The system control circuit 252 incorporates an A/D converter forconverting the outputs from the amplifiers 250 a and 250 b into digitalsignals. Reference numeral 256 denotes an anti-vibration ON/OFF switch,with which the user instructs the system control circuit 252 to turnon/off an anti-vibration mode. Reference numeral 258 denotes a memorysuch as an EEPROM or the like, which stores information unique to thecamera, for example, the pixel sizes, the numbers of extra pixels, theoutput video format, and the like of the image sensing element 242.

Reference numeral 260 denotes a CCD drive circuit which drives the imagesensing element 242 in accordance with a command from the anti-vibrationcontrol module 254 in the system control circuit 252 to read out adesired line portion; 262, a line memory for selecting an output imageportion in the line direction; and 264, a memory control circuit forcontrolling the line memory 262 in accordance with a command from theanti-vibration control module 254.

The image sensing element 242 converts an optical image formed by thelens unit 210 into an electrical signal, which is amplified by theamplifier 244. The amplified signal is supplied to the camera signalprocessing circuit 246. The camera signal processing circuit 246performs a known camera signal process for the output from the amplifier244, and outputs an NTSC video signal. The camera signal processingcircuit 246 generates a focus signal from the output from the amplifier244, and supplies it to the system control circuit 252. The systemcontrol circuit 252 transmits the focus signal supplied from the camerasignal processing circuit 246 to the lens control circuit 230 in thelens unit 210.

The anti-vibration control module 254 in the system control circuit 252calculates angular displacements by integrating the angular velocities(the outputs from the amplifiers 250 a and 250 b) detected by theangular velocity sensors 248 a and 248 b, calculates the pixel movementamount (nearly corresponding to f×tanθ) on the image sensing element 242resulting from vibration on the basis of the obtained angulardisplacements, i.e., vibration angles θ of the camera and the focallength f of the lens unit 210, and controls the CCD drive circuit 226and memory control circuit 264 to cancel that pixel movement as in thefirst embodiment.

The operation of the anti-vibration control module 254 will be explainedbelow with reference to FIGS. 14, 15, and 16. FIG. 14 is a flow chart ofthe initial setup routine of the anti-vibration control module 254. Thisroutine is executed once after power ON. FIG. 15 is a flow chart of aroutine for calculating vibration angle, and this routine is aninterrupt routine as in FIG. 3.

FIG. 16 is a flow chart showing the same process as in FIG. 4, which isexecuted once per field.

The control sequence shown in FIG. 14 will be explained first.

The above-mentioned information (e.g., the pixel sizes, the numbers ofextra pixels, the output video format, and the like) unique to thecamera is read from the memory 258 (step S41). The vertical andhorizontal pixel sizes of the image sensing element 242 in the uniqueinformation are respectively stored in memory areas α_(V) and α_(H), andthe numbers of vertical and horizontal extra pixels in the uniqueinformation are respectively stored in memory areas β_(V) and β_(H).When a PAL image sensing element is used in an NTSC camera, β_(V)=97 andβ_(H)=125. Furthermore, it is checked based on the output videoformation in the unique information if the camera complies with NTSC orPAL (step S42).

The interrupt frequency, i.e., the number of interrupts per field, inthe control sequence of the interrupt process (see FIG. 15) isdetermined in correspondence with the output video format (steps S43 andS44). The interrupt frequency is the sampling frequency of an angularvelocity signal, and is also the calculation frequency of angulardisplacement by various filter processes. The sampling frequency ispreferably constant independently of the output video format. That is,if the sampling frequency varies, the frequency characteristics invarious filter processes change. Sampling is preferably synchronous withthe field period to allow easy processes. In the third embodiment,information indicating the number of interrupts per field is set as thesampling frequency in a memory area K. When the sampling frequency isset at an integer multiple of the least common multiple (300 Hz) of thefield frequencies of NTSC and PAL (600 Hz in the third embodiment),identical frequency characteristics can be set for both NTSC and PAL inthe angular velocity signal process by a program.

In case of NTSC (step S42), the interrupt frequency is set at 600 Hz,and the number K of interrupts per field is set at 10 (step S43). Incase of PAL (step S42), the interrupt frequency is set at 600 Hz, andthe number K of interrupts per field is set at 12 (step S44).

An initial inter-communication with the lens control circuit 230 is madeto acquire focal length information (zoom lens position information)from the lens control circuit 230. Especially, focal lengths f_(T) andf_(W) at the most telescopic and wide-scopic positions and positioninformation are stored (step S46), thus ending this initial setupprocess.

FIG. 15 is a flow chart of the process for calculating angulardisplacements by integrating the angular velocity signals detected bythe angular velocity sensors 150 a and 150 b. This process is aninterrupt process executed at given periods by the system controlcircuit 252, i.e., at the frequency (600 Hz in the third embodiment (10times the field frequency in case of NTSC; 12 times the field frequencyin case of PAL)) determined in step S43 or S44. This frequencycorresponds to the sampling frequency of an angular velocity signal, andalso to the calculation frequency of angular displacement. An interruptsignal for this process can be generated by a known method. For example,clock signals are counted up or down, and upon counting clock signalscorresponding to 1/600 sec, an interrupt signal is generated. As in thesecond embodiment mentioned above, the system control circuit 252samples the angular velocity signals by converting them into digitalsignals using its internal A/D converter. In the third embodiment aswell, assume that the A/D converter operates in a scan mode, and alwaysconverts an input signal into a digital signal.

The DC component is removed from an angular velocity signal sampled bythe A/D converter (step S51), and the band of the AC component of theangular velocity signal is limited (step S52). The band limitation isthe same high-pass filter process as that for removing the DC component,except that its cutoff frequency is fixed in step S51 but is variable instep S52. By changing the cutoff frequency from the low- tohigh-frequency side, a desired band component is extracted. In the thirdembodiment, the cutoff frequency is increased to lower the suppressioncapability of anti-vibration during camera work such as panning or thelike, and is decreased to obtain a sufficient vibration correctioneffect in normal image sensing. In order to also prevent an unnaturalimage from being formed when a correction limit is reached in correctingvibration larger than the upper limit of the vibration correction range,the cutoff frequency in step S52 is adjusted. How to control the cutofffrequency will be explained later with reference to FIG. 16.

Angular displacement is calculated by integrating the band-limitedangular velocity signal (step S53). The calculated angular displacementcorresponds to a vibration angle acting on the camera main body. Avariable m indicating the number of times of vibration anglecalculations is incremented (step S54). If counter m equals the value ofmemory K (step S55), 0 is substituted in counter m (step S55), thusending the interrupt process; if counter m is not equal to the value ofmemory K (step S55), the process ends directly. In other words, after Kinterrupts have occurred per field period, counter m is reset to 0.

Steps S51 to S53 are executed for both the pitch and yaw directions.

The process shown in FIG. 16 is executed once per field. That is, theprocess shown in FIG. 16 is executed after the process shown in FIG. 15has been executed K times and before the next process starts, i.e., atthe end of the current field.

The control waits until m equals 0 (step S61). When the interruptprocess shown in FIG. 15 has been executed K times for the currentfield, m is reset to 0 (step S56). If m=0 (step S61), the current zoomlens position information is acquired via an inquiry to the lens controlcircuit 230 (step S62). The current focal length f is calculated basedon the previously acquired focal lengths f_(T) and f_(W) at the mosttelescopic and wide-scopic positions, and the current zoom lens position(step S63):

f=(f _(T) −f _(W))/zoom stroke×(most telescopic position−currentposition)  (8)

for zoom stroke=most telescopic position−most wide-scopic position. Thevibration correction amount is calculated based on the obtained currentfocal length f (step S64). As described above, the vibration correctionamount is given by f×tanθ using the vibration angle θ and the focallength f of the optical system. The target position coordinates (V₀, H₀)of the extraction range are calculated based on the calculated vibrationcorrection amount (step S65) by: $\begin{matrix}{{V_{0} = \quad {{{vertical}\quad {origin}\quad {position}} \pm {{number}{\quad \quad}{of}{\quad \quad}{pixels}{\quad \quad}{to}{\quad \quad}{be}\quad {moved}\quad {to}}}}{\quad \quad}} \\{\quad {{correct}\quad {pitch}\quad {vibration}\quad {angle}}} \\{= \quad {{\beta_{V}/2} \pm {\left( {- 1} \right) \times {pitch}\quad {vibration}{\quad \quad}{correction}{\quad \quad}{{amount}/\alpha_{V}}}}} \\{H_{0} = \quad {{{vertical}\quad {origin}\quad {position}} \pm {{number}{\quad \quad}{of}{\quad \quad}{pixels}{\quad \quad}{to}\quad {be}\quad {moved}}}} \\{\quad {{to}\quad {correct}\quad {yaw}\quad {vibration}\quad {angle}}} \\{= \quad {{\beta_{H}/2} \pm {\left( {- 1} \right) \times {yaw}\quad {vibration}\quad {correction}\quad {{amount}/\alpha_{H}}}}}\end{matrix}$

As a result, the numbers of pixels to be moved required for vibrationcorrection can be obtained.

The calculated vibration correction amount is normalized (step S66) by:

 normalized pitch vibration correction amount/α_(V)/θ_(V)/2×100(%)  (8)

normalized yaw vibration correction amount/α_(h)/β_(H)/2×100(%)  (9)

When the PAL CCD image sensing element 242 (582 vertical pixels×752horizontal pixels) is used in an NTSC camera, if 485 vertical linescomplying with NTSC are extracted from the PAL image sensing element,the number of horizontal pixels is 627 on the basis of the aspect ratio.Hence, the number of extra pixels β is 97×125 pixels.

The limiting threshold (cutoff frequency) is calculated from theobtained normalized vibration correction amount (step S67). The cutofffrequency has characteristics, as shown in, e.g., FIG. 12, and is raisedmore sharply as the vibration correction amount comes closer to themaximum correction limit.

An extraction command including the target position coordinates (V₀, H₀)calculated in step S65 as reference position coordinates of theextraction range is output to the CCD drive circuit 260 and memorycontrol circuit 264 (step S68). The flow then returns to step S61 toprepare for the next field, and the control waits until integrationrepeats itself E times.

In the third embodiment shown in FIG. 13, anti-vibration operation canbe realized using a common anti-vibration control program by onlyinitially setting a state unique to the camera and lens. Since thelimiting threshold of suppression capability of anti-vibration uponpanning can have normalized characteristics, a single anti-vibrationcontrol module can be used in different combinations of lens units andcamera main bodies. For example, even when lens units having differentlens characteristics are attached to a single camera body, or when alens unit of given characteristics is attached to camera main bodieshaving different capabilities, e.g., a camera that uses a high-densitytype, large-scale CCD image sensing element, or cameras using differentvideo formats such as NTSC, PAL, and the like, identical suppressioncapability of anti-vibration can be obtained. Especially, since a singleanti-vibration control module can be applied to every combinations oflenses and camera main bodies, a great cost reduction can be attained.

Advantages of First to Third Embodiments

As can be easily understood from the above description, according to thefirst to third embodiments, since the limiting threshold (e.g., cutofffrequency) of a vibration signal is determined in accordance with thevibration correction amount, the vibration signal can be continuouslylimited, and smooth mode transitions can be attained betweenanti-vibration control upon normal image sensing and that upon panning.In particular, since the limiting threshold changes in proportion to then-th power (n is an integer equal to or larger than 1) of the vibrationcorrection amount, flexible anti-vibration control (e.g., the vibrationcorrection amount is limited sharply, the limitation is minimized, orthe like) corresponding to situations can be set when the vibrationcorrection amount assumes a predetermined value. Also, by settingappropriate limiting characteristics in correspondence with the focallength, prevention of disturbance of the sensed image upon collisionagainst a vibration correction limit, and anti-vibration control forpanning can be implemented using identical parameters for bandlimitations.

Furthermore, by normalizing the vibration correction amount by the focallength and maximum correction limit, even in an image sensing apparatuswith a variable focal length and/or effective image aperture, a shiftand return to the panning mode can be smooth and natural by preparinglimiting characteristics for the normalized vibration correction amount.Uniform panning characteristics can be obtained by simple parametersetups irrespective of the anti-vibration scheme used.

Using an anti-vibration control program module, anti-vibration operationcorresponding to a variety of devices can be implemented by a commonanti-vibration control program by only initially setting a state uniqueto the camera and/or lens. Since the limiting threshold of suppressioncapability of anti-vibration upon panning can have normalizedcharacteristics, a value-added image sensing apparatus that canimplement a vibration correction function with identical characteristicsindependently of the types of lenses to be attached like in anexchangeable lens camera or the types of camera main bodies that mount agiven lens, such as a camera using a high-density type, large-scale CCDimage sensing element, or cameras of different video formats such asNTSC, PAL, and the like, can be provided. Especially, since a singlemodule can be applied to every combinations of camera systems, aninexpensive image sensing apparatus that can attain a great costreduction can be provided.

Fourth Embodiment

This embodiment will be explained below with reference to theaccompanying drawings. FIG. 17 is a block diagram showing thearrangement of an image sensing apparatus according to the fourthembodiment of the present invention. In this embodiment, a video cameraserving as an image sensing apparatus has an electronic anti-vibrationfunction.

As shown in FIG. 17, a lens unit has an inner focus type arrangement,and is composed of a first stationary lens 301, zoom lens 302, stop 303,second stationary lens 304, and focus lens 305. Light coming from thelens is imaged on an image sensing element 306 such as a CCD or thelike, and the output from the image sensing element 306 is amplified tooptimal level by an amplifier 307. The amplified signal is input to acamera signal processing circuit 308, and is converted into a standardtelevision signal. The camera shown in FIG. 17 has an electronic shakecorrection function, which is turned on/off by detecting the status of aswitch 317.

Angular velocity sensors 309 (pitch direction; detection means) and 310(yaw direction; detection means) detect the vibration angular velocitiesof the camera main body (image sensing element 306). The detectedvibration angular velocities are respectively amplified by amplifiers311 and 312, and are sampled by an A/D converter 315 a in ananti-vibration control microcomputer 315. The DC component is cut fromthe sampled angular velocity signal by a high-pass filter 315 b, andthat signal is integrated by an integral processor 315 d to be convertedinto an angular displacement. A vibration angle θ calculated by theintegral processor 315 d is corrected by a focal length correction unit315 e in correspondence with a focal length f of the optical system tocalculate a correction signal given by f·tanθ. A correction systemcontroller 315 h corrects vibration by moving an image in a directionopposite to the moving direction due to the vibration in correspondencewith the correction signal (corresponding to a pixel moving amount onthe image sensing element 306 due to the vibration) as an output signalof the focal length correction unit 315 e. Note that reference numeral315 c denotes a high-pass filter for band limitation. A limitationprocess controller 315 g controls the high-pass filter 315 c inaccordance with a normalized correction amount, which is normalized by acorrection amount normalization unit 315 f, and the output from thehigh-pass filter 315 b, thereby limiting the suppression capability ofanti-vibration upon panning. This limitation operation will be describedin detail later.

An image frame to be extracted for electronic anti-vibration control isan extraction frame 402 shown in, e.g., FIG. 18A. An image to beextracted will be explained below using FIG. 18A. FIG. 18A shows theimage sensing frame of the image sensing element 306, and a region 401corresponds to the entire image sensing frame. Of this region, a partialregion, e.g., the extraction frame 402 is extracted and undergoesdisplay or recording as the entire frame, as indicated by 404 in FIG.18B. The position of the extraction frame 402 is changed to correctshake by changing vertical and horizontal position coordinates (V₀, H₀)403. The position change range of the coordinates 403 is determined bythe differences of the numbers of horizontal and vertical pixels (to bereferred to as extra pixels hereinafter) between the entire imagesensing frame 401 and extraction frame 402. The coordinates 403 aredetermined to attain correction in such a manner that an origincoordinate position free from any shake is defined in advance, and theposition coordinates are changed in correspondence with the shake amountand direction.

In one method of extracting the extraction frame 402, an image of theentire image sensing frame 401 is temporarily stored in a field memory,only an image of the extraction frame 402 is enlarged to the size of theentire image sensing frame 401 while it is read out, and the enlargedframe 404 is displayed. In another method, a high-density,high-resolution type large-scale CCD is used as an image sensing so thatthe extraction region of the extraction frame 402 satisfies the numberof scanning lines required for a standard TV signal in advance. Sinceboth the former and latter methods require an expensive field memory andlarge-scale CCD, the fourth embodiment uses a versatile PAL CCD in anNTSC camera as in the above embodiments.

The PAL CCD has a high vertical pixel density. In the vertical scanningdirection for image extraction, a CCD drive circuit such as a timinggenerator or the like changes the number of lines that undergo fastdischarging within the range of extra lines exceeding the number oflines of NTSC in correspondence with any angular displacement caused byvibration, thereby changing the vertical image extraction position. Inthe horizontal scanning direction, when the relationship between thewrite start pixel position and read start pixel position with respect toa line memory is changed while enlarging image data in the horizontaldirection at a ratio corresponding to the aspect ratio, the horizontalframe position can be changed, thus realizing low-cost vibrationcorrection.

FIG. 17 has the aforementioned arrangement of the correction system. Asfor pixel movement in the vertical scanning direction, theanti-vibration control microcomputer 315 controls the CCD drive circuit316 to execute fast discharging control so as to extract a desiredscanning region. As for pixel movement in the horizontal scanningdirection, the line memory 313 and memory control circuit 314, whichsample a video signal processed by the camera signal processing circuit308, enlarge an image (attained by changing the memory read rate andreading out decimated pixel data) in correspondence with the aspectratio while varying the read position of the stored horizontal scanningimage in accordance with the correction pixel movement amount. Theobtained signal is fed back to the camera signal processing circuit 308,and is converted into a standard TV signal via, e.g., a color processand the like.

The anti-vibration control microcomputer 315 also controls the zoom lens302 and focus lens 305. A zoom switch unit 318 is a rotary switch, theresistance of which varies in correspondence with the pressure exerted.The anti-vibration control microcomputer 315 sends a drive command to amotor 319 via a motor driver 320 in response to a signal from the zoomswitch unit 318, thus moving the zoom lens 302 to zoom. Also, theanti-vibration control microcomputer 315 sends a drive command to amotor 321 via a motor driver 322 to maximize the level of a focus signalprocessed by the camera signal processing circuit 308, thus moving thefocus lens 305 to adjust the focus.

The anti-vibration control processed by the anti-vibration controlmicrocomputer 315 according to the fourth embodiment will be explainedbelow with reference to FIGS. 19 and 20.

An objective of the image sensing apparatus of the fourth embodiment isto attain natural shake correction that can smoothly switch control forlimiting the suppression capability of anti-vibration at the beginningor end of panning, and does not disturb camera works or sensed image.Hence, the features of the image sensing apparatus of the fourthembodiment are as follows.

1: The limiting amount is changed based o n predeterminedcharacteristics in accordance with the correction amount to change thecorrection effect level. In this case, different signals are used inenable determination upon enabling and disabling limiting operation.Especially, a correction signal is used in determination as to whetheror not the limiting operation is enabled, and a signal before bandlimitation or focal length correction is used in determination as towhether or not the limiting operation is disabled. In this way, hunchingof limiting operation is prevented , and uniform responsecharacteristics can be obtained independently of a change in fieldangle.

2: Upon strengthening and weakening the limiting operation, differentlimiting strength change rates are used to attain both quick responseand formation of a natural sensed image.

The description of the processing flows shown in FIGS. 19 and 20overlaps that of the processing of the anti-vibration controlmicrocomputer 315 shown in FIG. 17. Although the anti-vibration controlmicrocomputer 315 is illustrated as one building block in FIG. 17, it isimplemented by a program pre-stored in a ROM (not shown) in practice.Hence, the processing of the microcomputer 315 will be explained belowas that of a program.

The flow chart of FIG. 19 shows a process for calculating angulardisplacements by integrating angular velocity signals detected by theangular velocity sensors 309 and 310. This process is an interruptprocess executed at predetermined periods in response to an instructionfrom the anti-vibration control microcomputer 315. In this embodiment,the interrupt process is executed at a frequency 10 times the fieldfrequency, i.e., at 600 Hz in case of NTSC. This frequency correspondsto the sampling frequency of angular velocity signals, and thecalculation frequency of angular displacements. An interrupt start eventin the anti-vibration control microcomputer 315 occurs, for example,every time a counter that counts up (or down) oscillation clocks at apredetermined frequency division ratio reaches a count value thatmatches data corresponding to 1/600 sec. As has been described in FIG.17, the A/D converter 315 a in the anti-vibration control microcomputer315 samples the angular velocity signals. In this embodiment, assumethat the operation mode of the A/D converter 315 a is a scan mode, i.e.,the A/D converter always repeats A/D conversion, for the sake ofsimplicity.

When the interrupt process is started, the influence of the DC componentis removed by processing an A/D-sampled angular velocity signal via thehigh-pass filter in step S301. Step S302 is a process for limiting thefrequency band of an angular velocity signal of AC components, andincludes steps S302 a to S302 e (step S302 will be explained in detaillater in the description of FIG. 20). The process in step S302 isattained by substantially the same high-pass filter process as that instep S301 in practice, except that the cutoff frequency is fixed in stepS301 but is variable in step S302. By changing the cutoff frequency fromthe low- to high-frequency side, band limitation can be achieved.

How to control the cutoff frequency in step S302 will be explained laterin combination with the flow chart in FIG. 20. For example, the cutofffrequency is increased to lower the suppression capability ofanti-vibration during camera work such as panning or the like, and isdecreased to obtain a sufficient vibration correction effect in normalimage sensing. In order to prevent an unnatural image from being formedwhen a correction limit is reached in correcting vibration larger thanthe upper limit of the vibration correction range, the band limitationcontrol is also executed.

In step S303, angular displacement is calculated by integrating theband-limited angular velocity signal. The calculated angulardisplacement corresponds to a vibration angle θ acting on the cameramain body. In step S304, focal length correction is executed tocalculate the correction amount for vibration correction. As mentionedabove, the correction amount is given by f·tanυ in accordance with theangular displacement obtained in step S303, i.e., a vibration angle θand a focal length f of the optical system as in the first to thirdembodiments.

The processes in steps S303 to S307 form a processing routine forchecking if a vibration angle has been calculated 10 times per field:“m” as a register for storing a calculation count parameter isincremented (step S305), it is checked if “m=10” (step S306), and if theinterrupt has been generated 10 times, “m” is reset to “0” for the nextfield in step S307, thus ending this interrupt process. In steps S301,S302, S303, and S304 in FIG. 19, a vertical vibration signal isprocessed using a pitch angular velocity signal as the output from theangular velocity sensor 309, and a horizontal vibration signal isprocessed using a yaw angular velocity signal as the output from theangular velocity sensor 310.

The process shown in FIG. 20 is executed once per field, and is executedat a timing after the process shown in FIG. 19 has been executed 10times and before the next process starts, i.e., at the end of thecurrent field.

When the process is started, the control waits until “m=0” in step S401.If 10 interrupt processes have been executed in the current field and mis reset, the correction amount calculated in step S304 is normalized instep S402. The normalized correction amount is obtained by calculatingequations (4) and (5) in the second embodiment.

The arrangement of the fourth embodiment uses a PAL CCD (582 verticalpixels×752 horizontal pixels) in an NTSC camera. Upon extracting 485vertical lines complying with NTSC from this CCD, the number ofhorizontal pixels to be extracted is 627 in relation to the aspectratio. Hence, the number of extra pixels is 97 vertical pixels×125horizontal pixels. Since the sign of the correction direction changes incorrespondence with the direction of vibration, half the extra pixelsare used in normalization.

In step S403, the limiting amount for limiting correction capability iscalculated on the basis of the normalized correction amount calculatedin step S402. In the fourth embodiment, the limiting amount that limitscorrection capability corresponds to the cutoff frequency of thehigh-pass filter 315 c for band limitation. In step S403 a, a limitingtarget cutoff frequency f_(c) corresponding to the normalized correctionamount is calculated. This target cutoff frequency f_(c) is determinedby characteristics shown in FIG. 21.

FIG. 21 shows the characteristics of the limiting amount that limitscorrection capability, i.e., the cutoff frequency f_(c). The abscissaplots the normalized correction amount (%), and defines 100% correctionamount when correction is done using pixels half the extra pixels as amaximum correction limit. The ordinate plots the cutoff frequency f_(c)for band limitation as a parameter of the limiting amount. The cutofffrequency f_(c) has characteristics that change the cutoff frequency asa function of the square of the correction amount. In order to preventthe target correction value from reaching the correction limit and thesensed image from being disturbed as the vibration angle increases, thecharacteristics shown in FIG. 21 set to sharply raise the cutofffrequency as the correction ratio comes close to the maximum correctionlimit.

It is then checked in step S403 b if the target cutoff frequency f_(c)calculated in step S403 a is equal to or larger than the current cutofffrequency f_(c). If the target cutoff frequency f_(c)≧the current cutofffrequency f_(c), a “disable flag” is cleared. Note that the disable flagindicates whether to disable (stop) suppression of anti-vibrationcontrol (weakening of the anti-vibration effect), and is normally setwhen suppression becomes unnecessary, i.e., at the end of panning (whenYES is determined in step S403 c). The control is made in step S302 inFIG. 19 so as not to decrease the cutoff frequency f_(c) before thisflag is set. That is, the anti-vibration effect is inhibited from beingenhanced by weakening the limiting strength. This is to mainly preventhunching that may occur in anti-vibration limiting operation.

If it is determined in step S403 b that the target cutoff frequencyf_(c)<the current cutoff frequency f_(c), to determine whether or notpanning has ended it is checked in step S403 c if the angular velocitysignal output from the high-pass filter 315 b has become smaller than apredetermined value γ. Since the output signal from the high-pass filter315 b is a signal before band limitation or focal length correction,vibration of the camera can be directly detected independently of theimage sensing field angle, thus preventing hunching in limitingoperation and relaxing different response characteristics in units offield angles.

Note that the predetermined value γ is determined by measuring theoutput level of the high-pass filter 315 b at the end of panning inadvance.

If it is determined in step S403 c that the absolute value of the outputfrom the high-pass filter 315 b is equal to or larger than γ, the flowadvances to step S403 e; otherwise, the end of panning is determined,and the “disable flag” is set in step S403 d. Since correction is notlimited upon normal hand-held image sensing, the target and currentcutoff frequencies f_(c) become equal to each other in step S403 b, andthe “disable flag” remains cleared.

In such determination of image sensing situation upon panning, thelimiting operation is controlled in step S302 in FIG. 19. It is checkedin step S302 a in FIG. 19 if the target cutoff frequency f_(c)=thecurrent cutoff frequency f_(c). If YES in step S302 a, the flow advancesto step S303 without changing the cutoff frequency. If NO in step S302a, it is checked in step S302 b if the current cutoff frequency f_(c) issmaller than the target value. If YES in step S302 b, the current valuehas not reached the target value yet, and it is determined in step S403in FIG. 20 that limitation is to be strengthened. In this case, thecutoff frequency f_(c) is set to be larger by a predetermined value δthan the current value in step S302 e. If it is determined in step S302b that the target value f_(c) is smaller than the current value, it ischecked in step S302 c if the “disable flag” is set. If the disable flagis cleared, the current cutoff frequency f_(c) is not decreased but isheld, and the flow advances to step S303. If the disable flag=1, sincepanning has ended and the cutoff frequency can be decreased, the cutofffrequency is set to be smaller by a predetermined value ε than thecurrent value in step S302 d.

The process shown in FIG. 20 is executed at the field period, and theprocess shown in FIG. 19 is executed 10 times per field. In each controlcycle, an increase/decrease in cutoff frequency f_(c) is controlled inaccordance with the rates of change of the predetermined values δ and εused in steps S302 d and S302 e. The predetermined values δ and ε as thechange rates are determined to obtain, e.g., limiting strength changecharacteristics shown in FIGS. 22A to 22C.

FIG. 22A shows preferable change characteristics of the cutoff frequencyf_(c) at the beginning of panning, and exemplifies a case whereinpanning is started from a coordinate position 601 on the time axis. Atthe beginning of panning, the correction may reach the correction limitunless the cutoff frequency reaches the target value in a short responsetime. Since the image sensing frame is moving during panning, no imagedisturbance occurs even if the cutoff frequency is changed abruptly.Hence, the predetermined value δ for the beginning of panning is to beset at a relatively large value that can obtain a target limiting valuein a short period of time, as indicated by a curve 602.

FIG. 22B shows a preferable change in limiting amount when panning endsat a coordinate position 603.

Referring to FIG. 22B, since the image sensing frame is nearly in astill state after the coordinate position 603, an abrupt change incutoff frequency appears as a motion on the frame. On the other hand,when correction capability is increased immediately after the end ofpanning, the anti-vibration control begins to correct an angulardisplacement signal 1202 shown in FIG. 28C, and rebound occurs. In thefourth embodiment, in order to avoid such problems, the predeterminedvalue ε is determined to set slow response characteristics, i.e., toslowly change the cutoff frequency, as indicated by a curve 604 in FIG.22B.

In the fourth embodiment, the predetermined values δ and ε as the changerate data of the cutoff frequency have been explained as constants.However, the change rate data may be a time function or a function ofthe cutoff frequency. For example, by utilizing the fact that a changeis hardly observed when the cutoff frequency is high, suppressioncapability of anti-vibration is quickly raised to some extent to obtainhigh response characteristics, and is then raised gradually, asindicated by a characteristic curve 605 in FIG. 22C, thus preferablyrealizing natural anti-vibration control with respect to camera work.

Referring back to FIG. 20, in step S404 the target position coordinates(V₀, H₀) of the extraction position are calculated based on thecorrection signal (f*tanθ) calculated in step S303 in FIG. 19. Note thatthe target position is given by equations (1) and (2) of the thirdembodiment, thus obtaining the numbers of pixels to be moved invibration correction.

In step S405, a command that includes the target position coordinates(V₀, H₀) calculated in step S404 as the extraction position is output tothe CCD drive circuit 316 and memory control circuit 314, and the flowreturns to step S401 to prepare for the next field. Then, the controlwaits until integration repeats itself 10 times.

To restate, according to the fourth embodiment, since a limitingoperation determination means is provided, and different signals arereferred upon enabling and disabling limiting operation, hunching inlimiting operation, i.e., an unstable change in limiting effect, can beprevented. Also, since a signal before focal length correction isreferred to upon disabling the limiting operation, identical responsecharacteristics of limiting amount suppression can be set at the mostwide-scopic and telescopic positions. Furthermore, since differentlimiting strength change rates are used at the beginning and end oflimiting operation, the influence of a change in limiting strength onthe image and rebound can be suppressed while assuring high responsecharacteristics upon panning, thus realizing natural camera work atevery image sensing field angles.

Modification of Fourth Embodiment

The fourth embodiment has been explained with reference to thearrangement using a PAL CCD and line memory. Alternatively, correctionmay be done by controlling the position of an extracted image using afield memory, a large-scale or ultra high-resolution type CCD thatrequires no enlargement control may be used, or an optical correctionmeans may be used. In this embodiment, angular velocity sensors are usedas vibration detection means. Alternatively, acceleration sensors may beused. In such case, another integration process need only be addedinside or outside the anti-vibration control microcomputer.

The vibration angular displacement is calculated by software in thedescription of FIG. 17, but may be calculated by hardware. Thecharacteristics to be limited are determined as a function of thecorrection amount. However, the characteristics may be determined bycalculations using equations or may be pre-stored as a data table.

Fifth Embodiment

The fourth embodiment has exemplified the band limiting means of avibration signal by changing the cutoff frequency of a high-pass filteras a limiting means. The fifth embodiment uses a change in feedbackratio in integration as a limiting means. Note that the limiting meansis not limited to such specific means. For example, correctioncapability may be limited by changing the correction gain. In this case,even when the detection means is a motion vector detection means, thepresent invention can be applied.

FIG. 23 is a block diagram showing the arrangement of an image sensingapparatus according to the fifth embodiment. Note that the samereference numerals in FIG. 23 denote the same parts as those in thefourth embodiment, and a detailed description thereof will be omitted.

As shown in FIG. 23, the angular velocity sensors 309 (pitch direction)and 310 (yaw direction) detect the vibration angular velocities of thecamera main body. The detected vibration angular velocities arerespectively amplified by the amplifiers 311 and 312, and are sampled bythe A/D converter 315 a in the anti-vibration control microcomputer 315.The DC component is cut from the sampled angular velocity signals by thehigh-pass filter 315 b, and those signals are integrated by an integralprocessor 402 to be converted into angular displacements. Each vibrationangle θ calculated by the integral processor 402 is corrected by thefocal length correction unit 315 e in correspondence with a focal lengthf of the optical system to calculate a correction signal given byf*tanθ. The correction system controller 315 h corrects vibration bymoving an image in a direction opposite to its moving direction due tovibration in correspondence with the correction signal (corresponding toa pixel moving amount on the image sensing element 306 due to thevibration) as an output signal of the focal length correction unit 315e. Note that the integral processor 402 receives an integral feedbackamount determined by a limiting process controller 401. The limitingprocess controller 401 varies the feedback ratio to change the gaincharacteristics of the integral processor 402, thereby achieving bandlimitation. The correction signal calculated by the focal lengthcorrection unit 315 e is normalized by the correction amountnormalization unit 315 f, and the limiting process controller 401controls the integral processor 402 in accordance with the normalizedcorrection amount and the output from the high-pass filter 315 b, thuslimiting anti-vibration capability upon panning.

In the limiting control of the fifth embodiment, the integral feedbackratio is set as a change parameter in place of the cutoff frequencyf_(c) for band limitation in the flow charts shown in FIGS. 19 and 20 inthe fourth embodiment. Of the comparison and addition/subtraction stepsin the processes pertaining to the cutoff frequency f_(c) in step S302in FIG. 19 and step S403 in FIG. 20, the sense of the inequality isreversed, an addition and subtraction are replaced each other, and thelimiting parameter change characteristics (FIG. 21) in step S403 a inFIG. 20 are replaced by those shown in FIG. 24, thus obtaining the sameeffects as in the fourth embodiment. In addition, since the number oftimes of computations of the high-pass filter process in an interruptprocess can be smaller than that in the fourth embodiment, the load onthe anti-vibration control microcomputer can be reduced.

Sixth Embodiment

In the fourth and fifth embodiments, the output (1201 in FIG. 28B) fromthe high-pass filter 315 b is referred to upon determining whether ornot limitation is disabled. For this reason, if it is determined in stepS403 c in FIG. 20 that the absolute value level of the output from thehigh-pass filter 315 b is equal to or smaller than the predeterminedvalue γ, one of states 1204 or 1205 shown in FIG. 28B is detected (thestate 1204 is detected before the end of panning, and the state 1205 isdetected a predetermined period of time after the end of panning). Oneof these limiting disable timings is determined depending on the imagesensing situation such as panning speed, panning time, and the like. Forthis reason, when the limitation on the anti-vibration control isdisabled at the timing 1204, rebound readily occurs; when the limitationis disabled at the timing 1205, the response characteristics areimpaired.

In order to solve this problem, the sixth embodiment will explain amethod of determining using a signal before the high-pass filter processif limitation is disabled will be explained. Note that the DC componentof the signal before the high-pass filter process may change incorrespondence with the signal output level. Hence, the sixth embodimentwill also explain a method of reliably detecting the end of panningwithout being influenced by a change in DC component.

FIG. 25 is a block diagram showing the arrangement of an image sensingapparatus according to the sixth embodiment. Note that the samereference numerals in FIG. 25 denote the same parts as those in thefourth embodiment, and a detailed description thereof will be omitted.As shown in FIG. 25, the angular velocity sensors 309 (pitch direction)and 310 (yaw direction) detect the vibration angular velocities of thecamera main body. The detected vibration angular velocities arerespectively amplified by the amplifiers 311 and 312, and are sampled bythe A/D converter 315 a in the anti-vibration control microcomputer 315.The DC component is cut from the angular velocity signals by thehigh-pass filter 315 b, and the bands of those signals are limited bythe high-pass filter 315 c. The band-limited signals are converted intoangular displacements by integration in the integral processor 315 d.Each vibration angle θ calculated by the integral processor 402 iscorrected by the focal length correction unit 315 e in correspondencewith a focal length f of the optical system to calculate a correctionsignal given by f*tanθ. The correction system controller 315 h correctsvibration by moving an image in a direction opposite to its movingdirection due to vibration in correspondence with the correction signal(corresponding to a pixel moving amount on the image sensing element 306due to the vibration) as an output signal of the focal length correctionunit 315 e. Note that reference numeral 315 c denotes a high-pass filterfor band limitation. A limiting process controller 502 controls thehigh-pass filter 315 c in accordance with a normalized correction amountnormalized by the correction amount normalization unit 315 f and anoutput signal 501 from the A/D converter 315 a to limit anti-vibrationcapability upon panning. Using this signal 501, the end of panning isdetected. The output signal 501 upon panning changes like an angularvelocity signal shown in FIG. 29A. Using that output signal 501, the endof panning can be determined more reliably, and the timing of an endtime 1203 (FIG. 28C) can be detected more accurately.

The anti-vibration control flow of the sixth embodiment processed by theanti-vibration control microcomputer 315 will be explained below usingthe flow charts shown in FIGS. 26 and 27.

Note that the processing flows shown in FIGS. 26 and 27 havesubstantially the same processing contents as those in FIGS. 19 and 20.Hence, the same step numbers denote the same processes as those in thefourth embodiment, and a detailed description thereof will be omitted.

As a feature of the sixth embodiment, the end of panning is determinedwith reference to the angular velocity signal before DC componentremoval. Also, in order to detect only slope portions (panning start andend periods) of an angular velocity signal 1301 shown in FIG. 29Awithout being influenced by DC component variations of the referencesignal, it is checked with reference to a time change signal 1302 ofangular velocity if the signal level remains equal to or larger than apredetermined value k for a predetermined time τ or more, as shown inFIG. 29B, thereby determining the start or end of panning. With thisdetermination method, even when the DC component amount varies incorrespondence with the output level of the angular velocity sensor,such variation can be distinguished from the start or end of panningusing features, i.e., a small variation amount and long variation timeconstant.

In the fourth and fifth embodiments, the end of panning is determinedusing the output signal from the high-pass filter 315 b, and the samekind of signal as the time change signal 1302 of the angular velocityshown in FIG. 29B is obtained. However, the high-pass filter 315 b has acutoff frequency lower than 1 Hz and a long time constant so as not tocut off the shake signal but to cut off only the DC component. Bycontrast, the signal 1302 in FIG. 29B is defined as an angularacceleration signal generated by, e.g., differentiation with a shorttime constant (in the sixth embodiment, the signal is calculated as anangular velocity signal difference for a predetermined period of time).

The flow chart of FIG. 26 shows a process for calculating angulardisplacements by integrating angular velocity signals detected by theangular velocity sensors 309 and 310, which is similar to the processshown in FIG. 19. Note that this process is executed by a programpre-stored in a ROM (not shown.) in accordance with an instruction fromthe anti-vibration control microcomputer 315. The difference from theprocess shown in FIG. 19 is as follows. That is, before the high-passfilter process for removing the DC component in step S301, the outputsignal from the A/D converter 315 a is differentiated in step S1001 (inthis embodiment, a difference from the previous sampling value iscalculated), and it is checked in step S1002 if the absolute value ofthe calculated differential is larger than a predetermined value k (athreshold 1303 in FIG. 29B). If the absolute value is equal to orsmaller than k, counter n for measuring the panning shift time iscleared in step S1003. If the absolute value is larger than k, counter nis incremented in step S1004.

This counter n for measuring the panning shift time indicates theduration in which the angular acceleration signal level is equal to orhigher than k. For example, when the duration is equal to or longer thanτ indicated by 1304 in FIG. 29B, the start or end of panning isdetermined. As a feature of the sixth embodiment, limiting control isdone using the output from the unit 315 f at the beginning of panning inconsideration of the response characteristics, and whether or notlimiting control is disabled is determined at the end of panning withreference to the angular acceleration signal. Hence, if it is determinedin step S1101 in FIG. 27 that the count value of time measurementcounter n is larger than a predetermined value θ, the flow advances tostep S403 d to set the “disable flag” that permits to disable limitingcontrol. After that, correction capability is controlled to strengthenin step S302 in FIG. 26.

Advantages of Fourth to Sixth Embodiments

As described above, according to the fourth to sixth embodiments, animage sensing apparatus comprises detection means for detectingvibration, correction signal generation means for generating acorrection signal used for correcting a motion of an image due to thevibration detected by the detection means, correction means forcorrecting in accordance with the correction signal, and limiting meansfor limiting correction in correspondence with the correction signal.The limiting means includes limiting characteristics determination meansfor determining a limiting amount to obtain predeterminedcharacteristics in correspondence with a change in correction signal,band limiting means for limiting a band of the correction signal inaccordance with the determined limiting amount, and limiting operationdetermination means for determining using an input signal to the bandlimiting means and the correction signal whether or not the bandlimitation is to be made. Hence, hunching, i.e., an unstable change inlimiting amount, can be prevented, and natural camera work can berealized at every image sensing field angles not only at the beginningor in the middle of panning, but also at the end of panning.

Also, an image sensing apparatus comprises an image sensing element forconverting an optical image coming from a lens system into an electricalsignal, detection means for detecting vibration, correction signalgeneration means for generating a correction signal used for correctinga motion of an image due to the vibration detected by the detectionmeans in correspondence with a focal length of the lens system,correction means for correcting in accordance with the correctionsignal, limiting means for limiting correction to obtain predeterminedcharacteristics in correspondence with a change in correction signal,and limiting operation determination means for determining using asignal before amplification by a gain corresponding to the focal lengthby the correction signal generation means, and the correction signalwhether or not limiting operation is to be made. Thus, the same effectsas described above can be obtained.

Furthermore, an image sensing apparatus comprises an image sensingelement for converting an optical image coming from a lens system intoan electrical signal, detection means for detecting vibration,correction signal generation means for generating a correction signalused for correcting a motion of an image due to the vibration detectedby the detection means, correction means for correcting in accordancewith the correction signal, limiting means for limiting correction toobtain predetermined characteristics in correspondence with a change incorrection signal, and limiting operation determination means fordetermining using an output signal from the detection means, and thecorrection signal whether or not limiting operation is to be made. Thus,the same effects as described above can be obtained.

Seventh Embodiment

The seventh embodiment of the present invention will be explained belowwith the aid of FIGS. 30 to 39.

FIG. 30 is a block diagram showing the arrangement of an image sensingapparatus according to the seventh embodiment. The arrangement shown inFIG. 30 is substantially the same as that of the fourth embodiment,except for the details of the control microcomputer 315.

In the control microcomputer 315, a vibration angle θ calculated by anintegral circuit 315 d is corrected by an amount corresponding to thefocal length f of the optical system in a focal length correctioncircuit 315 e to calculate a correction signal given by f×tanθ. Acorrection system controller 315 h corrects vibration by moving an imagein a direction opposite to its moving direction due to vibration incorrespondence with the correction signal (corresponding to a pixelmoving amount on an image sensing element 106 due to vibration) as anoutput signal from the focal length correction circuit 315 e. Note thatreference numeral 315 c denotes a high-pass filter for band limitation(band-limiting HPF). The limiting process controller 315 g controls theband-limiting HPF 315 c in accordance with a normalized correctionamount obtained by normalizing the correction signal by a correctionamount normalization circuit 315 f, and the output from an HPF 315 b,thus limiting the suppression capability of anti-vibration upon panning.

The limiting operation will be described in detail later. As a featureof the seventh embodiment, filter coefficients (a, b) of theband-limiting HPF 315 c are stored in a table data memory 315 i as tabledata in units of cutoff frequencies, and each table data is formed toexhibit predetermined characteristics in correspondence with a change intable search look-up address. When anti-vibration control need belimited, the limiting strength is changed by selecting a filtercoefficient corresponding to a desired cutoff frequency from this table.

The anti-vibration control of the seventh embodiment is electronicallydone by shifting the read region. Hence, a description about this readregion will be omitted by quoting FIGS. 18A and 18B.

Also, a PAL CCD is used as the CCD for the same reason as in the fourthembodiment.

The anti-vibration control of the seventh embodiment processed by theanti-vibration control microcomputer 315 will be explained below withreference to FIGS. 31, 32, and 33.

The first object of the seventh embodiment is to realize naturalvibration correction that does not disturb camera work and the sensedimage by a simple limiting control method. As a feature of the seventhembodiment, band-limiting filter coefficients (a, b) are stored in thetable data memory 315 i as table data in units of cutoff frequencies,which data satisfy predetermined characteristics in correspondence witha change in table data search look-up address. Upon limiting control, afilter coefficient corresponding to a desired cutoff frequency isselected from this table to change the limiting strength.

The description of the control sequence of the seventh embodimentoverlaps that of the processing of the anti-vibration controlmicrocomputer 315 shown in FIG. 30. Although the anti-vibration controlmicrocomputer 315 is illustrated as one building block in FIG. 30, it isimplemented by a program in practice. Hence, the processing of themicrocomputer 315 will be explained below as that of a program usingFIGS. 31 and 32.

FIGS. 31 and 32 are flow charts showing the anti-vibration controlsequence, i.e., a process for calculating angular displacements byintegrating angular velocity signals detected by the angular velocitysensors 309 and 310. This process is an interrupt process executed atpredetermined periods in response to an instruction from theanti-vibration control microcomputer 315. In the seventh embodiment, theinterrupt process is executed at a frequency 10 times the fieldfrequency, i.e., at 600 Hz in case of NTSC. This frequency correspondsto the sampling frequency of angular velocity signals, and thecalculation frequency of angular displacements. An interrupt start eventin the anti-vibration control microcomputer 315 is produced, forexample, every time a counter that counts up (or down) oscillationclocks at a predetermined frequency division ratio reaches a count valuethat matches data corresponding to 1/600 sec. As has been described inFIG. 30, an A/D converter (not shown) in the anti-vibration controlmicrocomputer 315 samples the angular velocity signals. In thisembodiment, assume that the operation mode of the A/D converter 315 a isa scan mode, i.e., the A/D converter always repeats A/D conversion, forthe sake of simplicity.

Referring to FIGS. 31 and 32, when an interrupt process is started, theinfluence of the DC component is removed by processing an A/D-sampledangular velocity signal via the high-pass filter in step S401. Step S402is a process for limiting the frequency band of an angular velocitysignal of AC components, and includes steps S402 a to S402 e (step S402will be explained in detail later in the description of FIG. 33). Theprocess in step S402 is attained by substantially the same high-passfilter process as that in step S401 in practice, except that the cutofffrequency is fixed in step S401 but is variable in step S402. Bychanging the cutoff frequency from the low- to high-frequency side, bandlimitation can be achieved. How to control the cutoff frequency in stepS402 will be explained later in combination with the flow chart in FIG.33. For example, the cutoff frequency is increased to lower thesuppression capability of anti-vibration during camera work such aspanning or the like, and is decreased to obtain a sufficient vibrationcorrection effect in normal image sensing. In order to prevent anunnatural image from being formed when a correction limit is reached tocorrect vibration larger than the upper limit of the vibrationcorrection range, the band limitation control is also executed.

In step S403, angular displacements are calculated by integrating theangular velocity signals band-limited in step S402. Each calculatedangular displacement corresponds to a vibration angle θ acting on thecamera main body (image sensing apparatus main body). In step S404,focal length correction is executed to calculate the correction amountfor vibration correction. As mentioned above, the correction amountcalculated is given by f×tanθ in accordance with each angulardisplacement obtained in step S403, i.e., a vibration angle θ and afocal length f of the optical system.

The processes in steps S403 to S407 form a processing routine forchecking if the vibration angle has been calculated 10 times per field.More specifically, “m” as a register of a calculation count parameter isincremented in step S405, and it is checked in step S406 if “m=10” i.e.,if an interrupt has been generated 10 times. If an interrupt has beengenerated 10 times, “m” is reset to “0” in step S407 to prepare for thenext field, thus ending this process.

On the other hand, if it is determined in step S406 that the interrupthas not been generated 10 times yet, the control skips step S407, andends this process.

Note that in steps S401 to S404 a vertical vibration signal is processedusing a pitch angular velocity signal as the output from the angularvelocity sensor 309, and a horizontal vibration signal is processedusing a yaw angular velocity signal as the output from the angularvelocity sensor 310.

The process shown in the flow of FIG. 33 is executed once per field, andis executed at a timing after the process shown in FIGS. 31 and 32 hasbeen executed 10 times and before the next process starts, i.e., at theend of the current field.

Referring to FIG. 33, when the process is started, the control waitsuntil “m=0” in step S501. If 10 interrupt processes have been executedin the current field and m is reset, the correction amount calculated instep S404 in FIGS. 31 and 32 is normalized in step S502.

The normalized pitch and yaw correction amounts are calculated byequations (6) and (7) in the first modification mentioned above.

The image sensing apparatus of the seventh embodiment uses a PAL CCD(582V×752H) in combination with an NTSC camera. Upon extracting 485vertical lines complying with NTSC from this CCD, the number of verticalpixels to be extracted is 627 in relation to the aspect ratio. Hence,the extra pixel block is defined by 97V×125H. Since the sign of thecorrection direction changes in correspondence with the direction ofvibration, half the extra pixels are used in normalization.

The next step S503 is a process for calculating a look-up address usedfor reading out a limiting amount that limits correction capability fromthe memory 315 i, on the basis of the normalized correction amountcalculated in step S502, and includes steps S503 a to S503 e. Note thatthe limiting amount includes filter coefficients (a, b) corresponding tothe cutoff frequency of the band-limiting high-pass filter 315 c.

In step S503 a, a look-up address A_(n) corresponding to the normalizedcorrection amount is calculated. The look-up address A_(n) is used forlooking up a data table in the data table memory 315 i, and a limitingtarget cutoff frequency is stored at that address. This target look-upaddress A_(n) is determined by characteristics shown in FIG. 34.

FIG. 34 shows the characteristics of the target look-up address as afunction of the normalized correction amount. In FIG. 34, the abscissaplots the normalized correction amount, which assumes 100% whencorrection is done using all pixels half the extra pixels thatcorrespond to a maximum correction limit. The ordinate plots the look-upaddress used for searching the data table in the data table memory 315i. This embodiment uses characteristics that proportionally change thelook-up address when the correction amount is 30% or more. When thecorrection amount is 100%, a look-up address “128” is selected, and thecutoff frequency at that time is 6 Hz. This is because the vibrationfrequency is 5 Hz or less in practice, as has been described previouslyin the prior art.

FIG. 35 shows the relationship between the look-up address A_(n) andcutoff frequency f_(c). In FIG. 35, the abscissa plots the normalizedcorrection amount, and the ordinate plots the cutoff frequency.Referring to FIG. 35, a higher cutoff frequency is set as the look-upaddress becomes larger. Then, filter coefficients (a, b) correspondingto the cutoff frequency are read out from the data table in the datatable memory 315 i. The relationship between the look-up address andcutoff frequency is defined by characteristics that change the cutofffrequency as a function of the square of the address A_(n). In order toalso prevent the sensed image from being disturbed upon reaching acorrection limit as the vibration angle becomes larger, thecharacteristics shown in FIG. 34 are set to sharply increase the cutofffrequency as the correction ratio comes closer to the maximum correctionlimit.

FIG. 36 shows an example of the data table in the data table memory 315i. The data table memory 315 i stores two kinds of variables, i.e.,filter coefficients a(A_(n)) and b(A_(n)), as data sequences incorrespondence with the look-up addresses, and filter coefficientsa(A_(n)) and b(A_(n)) are prepared in advance by calculations inassociation with required cutoff frequencies. Note that the data tableshown in FIG. 36 stores multiplication coefficients that achieve thehigh-pass filter characteristics shown in FIG. 21, and FIG. 37 shows thearrangement of a filter that gives such characteristics.

Referring to FIG. 37, reference numeral 801 denotes input data (S_(in));806, a storage unit for data (Z⁻¹) in the previous sampling period; and807, a first multiplier circuit which multiplies the data (Z⁻¹) in theprevious sampling period by a filter coefficient a(A_(n)). Referencenumeral 802 denotes a first adder circuit which adds the input data Sin801 to the product of the filter coefficient a(A_(n)) and the data (Z⁻¹)in the previous sampling period in the storage unit 806. Let c be thisoutput data. Reference numeral 808 denotes an inverter circuit whichinverts the sign of the data (Z⁻¹) in the previous sampling period inthe storage unit 806. Reference numeral 803 denotes a second addercircuit which adds the output data c to the sign-inverted data (-Z⁻¹) inthe previous sampling period. Reference numeral 804 denotes a secondmultiplier circuit for multiplying (c-Z⁻¹) by a filter coefficientb(A_(n)). Reference numeral 805 denotes final output data (S_(out)) ofthe high-pass filter, i.e., the output from the second multipliercircuit 804. On the other hand, the intermediate data (output data) c isstored as the data (Z⁻¹) in the previous sampling period in the storageunit 806 for the next computation. By executing a series of computationsat a predetermined cycle, a high-pass filter is implemented.

As shown in FIG. 37, the look-up address is determined in correspondencewith a change in correction amount upon panning, and the filtercoefficients a(A_(n)) and b(A_(n)) are selected from the data table inthe data table memory 315 i in accordance with the determined address,thus changing the cutoff frequency.

Referring back to FIG. 33, it is checked in step S503 b if the targetlook-up address A_(n) calculated in step S503 a is equal to or largerthan the current look-up address A_(n). If the target look-up addressA_(n) is equal to or larger than the current look-up address A_(n), the“disable flag” is cleared in step S503 e, and the flow then advances tostep S504. Note that the disable flag is the same as that used in otherembodiments, and is set upon completion of panning. Before this flag isset, control is made in step S502 in FIGS. 31 and 32 so as not to lowerthe cutoff frequency, i.e., not to decrease the look-up address A_(n).This process inhibits the anti-vibration effect from being enhanced byweakening the limiting strength, so as to prevent hunching of limitingoperation.

On the other hand, if it is determined in step S503 b that the targetlook-up address A_(n) is smaller than the current look-up address A_(n),to determine whether or not panning has ended it is checked in step S503c if the angular velocity signal output from the high-pass filter 315 bis smaller than a predetermined value γ. Since the output signal fromthe high-pass filter 315 b is a signal before band limitation or focallength correction, vibration of the camera can be directly detectedindependently of the image sensing field angle, thus preventing hunchingin limiting operation and relaxing different response characteristics inunits of field angles.

Note that the predetermined value γ is determined by measuring theoutput level of the high-pass filter 315 b at the end of panning inadvance, as in the above embodiment.

If it is determined in step S503 c that the absolute value of the outputfrom the high-pass filter 315 b is equal to or larger than γ, it isdetermined that panning is continuing and the flow advances to step S503e; otherwise, the “disable flag” is set in step S503 d. Since correctionis not limited upon normal hand-held image sensing, the target andcurrent look-up addresses A_(n) are equal to each other in step S503 b,the “disable flag” remains cleared.

In such determination of the image sensing situation upon panning, thelimiting operation is controlled in step S402 in FIGS. 31 and 32. It ischecked in step S402 a in FIG. 31 if the target look-up address A_(n) isequal to the current look-up address A_(n). If YES in step S402 a, theflow advances to step S402 h to read out the filter coefficientsa(A_(n)) and b(A_(n)) corresponding to the same cutoff frequency as thecurrent one from the table and to set these coefficients in theband-limiting HPF315 c. Then, the flow advances to step S403.

On the other hand, if it is determined in step S402 a that the targetand current look-up addresses A_(n) are not equal to each other, it ischecked in step S402 b if the current look-up address A_(n) is smallerthan the target look-up address A_(n). If the current look-up addressA_(n) is smaller than the target look-up address A_(n), the currentlook-up address A_(n) has not reached the target look-up address A_(n)yet, and it is determined in step S503 in FIG. 33 that the limitingstrength is to be increased. In such case, the look-up address A_(n) isset to be larger by a predetermined value δ than the current value instep S402 g in FIG. 32, and the flow then advances to step S402 h.

On the other hand, if it is determined in step S402 b that the targetlook-up address A_(n) is smaller than the current look-up address A_(n),it is checked in step S402 c if the “disable flag” is set. If thedisable flag is cleared, the current look-up address value is held notto lower the cutoff frequency, and the flow advances to step S402 h.

If it is determined in step S402 c that the disable flag=1, sincepanning has ended and the cutoff frequency can be decreased, counter C0is incremented in step S402 d and it is then checked in step S402 e ifcounter C0 assumes an even value. If counter C0 assumes not an evenvalue but an odd value, the flow advances to step S402 h; otherwise, thelook-up address is set to be smaller by a predetermined value ε than thecurrent value in step S402 f. Counter C0 delays the change cycle of thelook-up address A_(n) compared to that at the beginning of panning so asto reduce any change in cutoff frequency at the end of panning. In stepS402 h, the filter coefficients corresponding to the re-set look-upaddress A_(n) are read out to update the setups of the band-limiting HPF315 c.

The process shown in FIG. 33 is executed at a field period, and theprocess shown in FIGS. 31 and 32 is executed 10 times per field. In eachcontrol cycle, an increase/decrease in look-up address A_(n) iscontrolled in accordance with the rates of change of the predeterminedvalues δ and ε used in steps S402 f and S402 g. The predetermined valuesδ and ε as the change rates are determined to obtain, e.g., limitingstrength change characteristics shown in FIGS. 38 and 39. In FIGS. 38and 39, the abscissa plots time, and the ordinate plots the cutofffrequency.

FIG. 38 shows the change characteristics of the cutoff frequency at thebeginning of panning (image sensing), and exemplifies a case whereinpanning is started from time 901. At the onset of panning, thecorrection may reach a correction limit unless the cutoff frequencyreaches the target value in a short response time. Since the imagesensing frame is moving during panning, no image disturbance occurs evenif the cutoff frequency is changed abruptly. Hence, the predeterminedvalue δ assumes a relatively large value to obtain a target limitingamount within a short period of time like at time 902.

FIG. 39 shows a case wherein panning ends at time 903. Since the imagesensing frame is nearly in a still state after time 903, an abruptchange in cutoff frequency appears as a motion on the frame. On theother hand, when correction capability is increased immediately afterthe end of panning, rebound occurs. To solve these problems, thepredetermined value ε is determined to lower the responsecharacteristics at the end of panning and to slowly change the cutofffrequency, as indicated by a curve 904. As a result, the change cycle istwice that at the beginning of panning.

In the seventh embodiment, the change cycle of the look-up address atthe end of panning is twice that at the beginning of panning. However,the present invention is not limited to such specific change cycle. Forexample, the change cycle need only be set to prevent rebound and toprevent a change in band limitation from forming an unnatural image.

In the method of the seventh embodiment, the predetermined values δ andε are constants, and the look-up address changes linearly. With thismethod, the cutoff frequency as a band-limiting parameter consequentlychanges in accordance with the characteristics shown in FIG. 35, and thelimiting target cutoff frequency corresponding to the correction amountand the change locus to the target cutoff frequency can be changedaccording to identical characteristics. Hence, a change locus 162 shownin FIG. 10A does not form, and an image can be prevented from beingdisturbed by a change in cutoff frequency.

In addition, the control process can be implemented by simplecomputations, i.e., addition/subtraction of the look-up addresses, and asystem can be configured using a lower-cost microcomputer.

Referring back to FIG. 33, in step S504 target position coordinates (V₀,H₀) of the extraction position are calculated based on the correctionsignal (f×tanθ) calculated in step S503 in FIGS. 31 and 32. Note thatthe target position is given by equations (1) and (2) of the thirdembodiment, thus obtaining the numbers of pixels to be moved invibration correction.

In step S505, a command that includes the target position coordinates(V₀, H₀) calculated in step S504 as the extraction position is output tothe CCD drive circuit 316 and memory control circuit 314, and the flowreturns to step S501 to prepare for the next field. Then, the controlwaits until integration repeats itself 10 times.

As described above, according to the seventh embodiment, a plurality ofband-limiting data that define predetermined characteristics incorrespondence with the cutoff frequency used in limiting operation areprepared as a ROM table, the table data look-up address is controlled incorrespondence with the vibration correction amount, and the amount andcycle of a change in table data look-up address are controlled inaccordance with the current camera operation state. Hence, aninexpensive microcomputer can be selected, and parallel processes withother processes can be made. In addition, the response characteristicsof the limiting operation at the beginning of panning can be improved,and rebound that is likely to occur at the end of panning can beprevented. Furthermore, the limiting strength and its change rate can bedetermined using identical characteristics, thereby preventing a motionof an image due to a change in limiting strength upon panning, andrealizing natural camera work at every image sensing field angles.

Note that the seventh embodiment has been explained with reference to anarrangement using a PAL CCD and line memory. Alternatively, correctionmay be done by controlling the position of an extracted image using afield memory, a large-scale or ultra high-resolution type CCD thatrequires no enlargement control may be used, or an optical correctionmeans may be used.

In the seventh embodiment, angular velocity sensors are used asvibration detection means. Alternatively, acceleration sensors may beused. In such case, another integration process need only be addedinside or outside the anti-vibration control microcomputer.

In the seventh embodiment, the vibration angular displacement iscalculated by software, but may be calculated by hardware.

Moreover, the seventh embodiment has exemplified band limitation using ahigh-pass filter as a limiting means. Alternatively, integral feedbackcoefficients and gain coefficients in the integral circuit 315 d may beprepared as table data, and band limitation may be implemented byintegration.

To restate, according to the image sensing method and apparatus of theseventh embodiment, since data are selected from a band-limiting datatable that defines limiting operation to obtain predeterminedcharacteristics, a plurality of data that pertain to band limitation canbe simultaneously acquired, and the computation time required forcalculating the data can be omitted. Hence, even in anti-vibrationcontrol processed at high speed, an inexpensive microcomputer can beselected, and parallel processes with other processes (focus adjustment,exposure adjustment, and the like) can be done, thus selecting a simplesystem.

According to the image sensing method and apparatus of the seventhembodiment, since the amount and cycle of a change in look-up addressused for searching the data table are changed in correspondence with thepertinent image sensing situation, limiting operation can be quicklydone at the beginning of panning, and rebound that readily takes placeat the end of panning can be prevented.

Furthermore, according to the image sensing method and apparatus of theseventh embodiment, especially, since limiting control determines thetarget value by selecting data from the data table with predeterminedcharacteristics and determines the look-up address so it reaches thetarget value, the limiting strength and its change rate can bedetermined using identical characteristics. Hence, any movement of animage due to a change in limiting strength upon panning can beprevented, and natural camera work can be realized at every imagesensing field angles.

Moreover, according to a storage medium of the seventh embodiment, theaforementioned image sensing apparatus can be smoothly controlled.

Eighth Embodiment

The eighth embodiment of the present invention will be described belowwith reference to the drawings. FIG. 40 is a block diagram showing thearrangement according to the eighth embodiment of the present invention,in which a video camera has an optical anti-vibration function. Thehardware arrangement of the image sensing apparatus of the eighthembodiment is substantially the same as that of the fourth embodiment,except for the details inside an anti-vibration control microcomputer315.

A lens unit has an inner focus type arrangement, and is composed of afirst stationary lens 301, zoom lens 302, stop 303, second stationarylens 304, and focus lens 305. Light coming from the lens is imaged on animage sensing element 306 such as a CCD or the like, and the output fromthe image sensing element 306 is amplified to optimal level by anamplifier 307. The amplified signal is input to a camera signalprocessing circuit 308, and is converted into a standard televisionsignal. The camera shown in FIG. 40 has an optical shake correctionfunction, which is turned on/off by detecting the status of a switch324.

Angular velocity sensors 309 (pitch direction) and 310 (yaw direction)serving as vibration detection means detect the vibration angularvelocities of the camera main body. The detected vibration angularvelocities are respectively amplified by amplifiers 311 and 312, and aresampled by A/D converter 315 a ₁ and 315 a ₂ in an anti-vibrationcontrol microcomputer 315. Angular velocity signals from which the DCcomponents are cut by high-pass filters 315 b ₁ and 315 b ₂ areband-limited by band-limiting HPFs 315 c ₁ and 315 c ₂. These angularvelocity signals are then converted into angular displacements byintegration in integrators 315 d ₁ and 315 d ₂. Vibration angles θcalculated by the integrators 315 d ₁ and 315 d ₂ are corrected by anamount corresponding to a focal length f of the optical system in focallength correction units 315 e ₁ and 315 e ₂ to calculate correctionsignals given by f*tanθ.

A correction system controller 315 h corrects vibration by moving theshift lens 304 in the pitch and yaw directions perpendicular to theoptical axis, so that the correction signals (moving amounts of a sensedimage due to vibration on the image sensing element) as the outputsignals from the focal length correction units 315 e ₁ and 315 e ₂ in adirection opposite to the image moving direction due to the vibration.These correction system controller 315 h and shift lens 304 construct acorrection means that corrects motion of an image. The shift lens 304 iscontrolled as follows. That is, with respect to a correction targetsignal output from the microcomputer 315, an adder 316 compares aposition signal of the shift lens 304 (i.e., a position signal obtainedby amplifying a detection signal of an encoder 313 to a predeterminedlevel by an amplifier 314) with a correction target from the controller315 h, and a drive signal is output to a motor 318 via a motor driver317 to obtain zero difference, thus loop-controlling the position of thelens 304 to match a target position. The correction system controller315 h, shift lens 304, and its control system construct a correctionmeans that corrects motion of an image due to vibration.

Note that a limiting process controller 315 g controls the band-limitinghigh-pass filters 315 c ₁ and 315 c ₂ in accordance with normalizedcorrection amounts obtained by normalizing the correction signals bycorrection amount normalization units 315 h ₁ and 315 h ₂, and vibrationfrequencies detected by vibration frequency detectors 315 f ₁ and 315 f₂, thereby limiting the suppression capability of anti-vibration uponpanning.

The limiting process controller 315 g has a vibration statediscrimination unit 315 g ₁ and stores panning characteristic data(limiting characteristics) 315 g ₂. In accordance with the normalizedcorrection amount signals and vibration frequency signals input to thelimiting process controller 315 g, the vibration state discriminationunit 315 g ₁ discriminates the image sensing situation, and the limitingprocess controller 315 g selects panning characteristics optimal to thatimage sensing condition from the panning characteristic data 315 g ₂ andcontrols the cutoff frequency as a band-limiting parameter of theband-limiting HPFs 315 c ₁ and 315 c ₂ in accordance with the selectedcharacteristics, thereby determining the limiting amount and limitingsuppression capability of anti-vibration upon panning. The limitingamount determination process corresponds to a limiting means in theappended claims of the present invention. The limiting operation will beexplained in detail later.

The anti-vibration control microcomputer 315 also controls the zoom lens302 and focus lens 305. In response to a signal from a rotary zoomswitch 318, the resistance of which changes in correspondence with thepressure inflicted, the anti-vibration control microcomputer 315 sends adrive command to a motor 319 via a motor driver 320, thus moving thezoom lens 302 to zoom. Also, the anti-vibration control microcomputer315 sends a drive command to a motor 321 via a motor driver 323 tomaximize the level of a focus signal, which is processed by a camerasignal processing circuit 308 and has as a focus evaluation value ahigh-frequency component or the like in a video signal, thus moving thefocus lens 305 to an in-focus point to adjust the focus.

The flow of anti-vibration control of the eighth embodiment processed bythe anti-vibration control microcomputer 315 will be explained belowusing FIGS. 41 and 42. The objective of the present invention is toimplement smooth panning and high suppression capability ofanti-vibration by selecting optimal anti-vibration characteristics incorrespondence with each image sensing situation. For this purpose, theeighth embodiment comprises means, having a plurality of pre-storedpanning characteristics (limiting characteristics that limit thesuppression capability of anti-vibration), for determining the imagesensing situation, automatically selecting optimal panningcharacteristics in correspondence with that image sensing situation, andexecuting anti-vibration control.

The plurality of pre-stored panning characteristics containcharacteristics which set different panning characteristics in the pitchand yaw directions, so as to cope with an image sensing situationdifferent from normal hand-held image sensing.

The following description of the processing flow overlaps that of theprocessing of the anti-vibration control microcomputer 315 shown in FIG.40. Although the anti-vibration control microcomputer 315 is illustratedas one building block in FIG. 40, it is implemented by a program inpractice. Hence, the processing of the microcomputer 315 will beexplained below as that of a program.

The flow chart shown in FIG. 41 is a process for calculating angulardisplacements by integrating angular velocity signals detected by theangular velocity sensors 309 and 310 to calculate the correction amountsand limiting amounts. However, for the sake of simplicity, the flowchart in FIG. 41 shows a process routine executed for one angularvelocity sensor, and the same process as that in FIG. 41 applies to theother angular velocity sensor output. The process shown in FIG. 41 is aninterrupt process executed by the anti-vibration control microcomputer315 at a predetermined period, e.g., at a frequency of 1 kHz. Aninterrupt start event in the anti-vibration control microcomputer 315 isproduced, for example, every time a counter that counts up (or down)oscillation clocks at a predetermined frequency division ratio reaches acount value that matches data corresponding to 1 msec. As has beendescribed in FIG. 40, the A/D converter in the anti-vibration controlmicrocomputer 315 samples the angular velocity signals. In the eighthembodiment, assume that the operation mode of the A/D converter is ascan mode, i.e., the A/D converter always repeats A/D conversion, forthe sake of simplicity.

The interrupt process is started in step S601, and the influence of theDC component is removed by processing the A/D-converted angular velocitysignal via the high-pass filter in step S602 to extract only thevibration component. In step S603, the vibration frequency is detectedfrom the DC-cut angular velocity signal. Frequency detection in thevibration frequency detectors 315 f ₁ and 315 f ₂ is done by convertingthe angular velocity signal into an angular displacement by integration,and detecting the number of times the displacement direction of theangular displacement signal changes within a predetermined period oftime, i.e., i.e., the vibration frequency.

The vibration frequency detected in step S603 is used in vibration statediscrimination in step S608 (to be described later), thus discriminatingthe image sensing condition.

The process in step S604 limits the frequency band of the angularvelocity signal as the vibration component, i.e., the AC component. Inpractice, this process is substantially the same high-pass filterprocess in step S602, except that the cutoff frequency is fixed in stepS602 but is variable in step S604, and the cutoff frequency is changedin accordance with instruction information from the limiting processcontroller 315 g. By changing the cutoff frequency from the low- tohigh-frequency side, band limitation can be achieved. The cutofffrequency is controlled as follows. That is, when the cutoff frequencyis increased to lower the suppression capability of anti-vibrationduring camera work such as panning or the like, the centering strengthof the position of the shift lens 304 is increased to realize smoothcamera work. On the other hand, the cutoff frequency is decreased to theneighborhood of shake frequency to obtain a sufficient shake removaleffect in normal image sensing.

In order to also prevent an unnatural image from being formed when thecorrection limit is reached upon correcting vibration beyond the upperlimit of the correction range, the band limitation control is executed.

In step S605, angular displacement is calculated by integrating theband-limited angular velocity signal by the integrator 315 d ₁ or 315 d₂. The calculated angular displacement corresponds to a vibration angleacting on the camera main body.

In step S606, the correction amount is calculated. The correction amountis given by f*tanθ in accordance with the angular displacement obtainedby the process in step S605, i.e., the vibration angle θ and the focallength f of the optical system. In step S607, the correction amountcalculated in step S606 is normalized by a maximum correction limit (themovement limit of the shift lens 304). The normalized correction amountis calculated by equations (1) and (2) above.

Step S608 is a vibration state discrimination routine for discriminatingthe image sensing situation on the basis of the normalized correctionamount calculated in step S607 and the vibration frequency detected instep S603, and the vibration state discrimination unit 315 g ₁ executesa discrimination process shown in the flow chart of FIG. 42.

In step S608, the vibration state is discriminated by the process insteps S701 to S716 shown in FIG. 42. It is checked in step S701 if thevibration frequency in the pitch direction detected in step S603 isequal to or higher than 20 Hz, i.e., if on-vehicle image sensing forsensing an image on, e.g., a vehicle on the move is done.

In case of on-vehicle image sensing, at least vibration in the pitchdirection is caused by those resulting from vibrations of an engine ortraveling, and frequency components of 20 Hz or higher dominate. On theother hand, in case of normal hand-held image sensing, the vibrationfrequency ranges from about 1 Hz to 5 Hz.

If YES in step S701, it is checked in step S702 if the value of counterM is equal to or larger than a predetermined value ρ. If M≦ρ, counter Mis incremented in step S703.

Hence, when vibration of 20 Hz or more continues, the value of counter Mbecomes larger than ρ, and in that case, the flow advances to step S714to determine on-vehicle image sensing.

Note that counter M measures the duration of vibration of 20 Hz orhigher, or the duration of vibration less than 20 Hz. If this counter ismonitored and predetermined duration ρ has elapsed, it is determinedthat “on-vehicle image sensing is done” or “on-vehicle image sensing isnot done”.

In case of hand-held image sensing, NO is determined in step S701, andthe flow advances to step S704. It is checked in step S704 if the valueof counter M is zero. If NO In step S704, counter M is decremented instep S705, and the control enters the process starting from step S706;otherwise, i.e., if M=0, the flow advances to step S706.

In steps S706 to S711, a tripod image sensing discrimination process isexecuted. In steps S706 and S707, it is checked if the vibrationfrequency in the pitch/yaw direction is 1 Hz or less and the vibrationamplitude (corresponding to the correction amount normalized in stepS607) is smaller than a predetermined value ω. If vibration has a lowfrequency and very small amplitude, it is checked in step S708 if thevalue of counter N is larger than a predetermined value γ. If YES instep S708, tripod image sensing is determined in step S715.

If NO in step S708, counter N is incremented in step S709, and the flowadvances to step S712. Note that counter N measures the duration ofvibration “which has a frequency of 1 Hz or less and very smallamplitude” or vibration “which has neither a frequency of 1 Hz or lessnor very small amplitude”. If this counter is monitored andpredetermined duration γ has elapsed, it is determined that “tripodimage sensing is done” or “tripod image sensing is not done”.

Note that tripod image sensing has a feature that a still state isdetected in both the pitch and yaw directions, and both the vibrationfrequency and amplitude assume values in the neighborhood of zero, anddiscrimination is done based on such feature. In this embodiment, thediscrimination frequency is set at 1 Hz and the discrimination amplitudeis set at ω. However, the present invention is not limited to suchspecific values, and these values can be determined in correspondencewith the sensitivity (gain) of the angular velocity sensor used.

On the other hand, normal hand-held image sensing has a featuredifferent from that of tripod image sensing, since the vibrationfrequency ranges from around 1 Hz to 5 Hz and the vibration has anamplitude equal to or larger than the predetermined value.

If the vibration frequency is larger than 1 Hz (step S706), or if thevibration amplitude is equal to or larger than the predetermined value ω(step S707), it is checked in step S710 if the value of counter N iszero. If NO in step S710, counter N is decremented in step S711, and theflow advances to step S712; otherwise, i.e., if N=0, the flow directlyadvances to step S712.

It is checked in steps S712 and S713 if counters M and N are zero. Ifone of the values of these counter is not zero, the control exits thisprocess, and advances to step S609 in FIG. 41.

On the other hand, if the values of both the counters are zero, normalhand-held image sensing is determined in step S716.

Steps S712 and S713 are set to keep the previous mode setups even whenthe on-vehicle, tripod, or hand-held mode instantaneously shifts to avibration state that indicates another mode, unless the new vibrationstate continues for an extended period of time.

This is to smoothly attain mode shift by providing hysteresis to modeswitching.

Note that the flow advances from step S703 or S705 to the processstarting from step S706 to achieve quick shift response between theon-vehicle mode and tripod mode. For example, with this process, uponon-vehicle image sensing using a camera set on a tripod, mode shift“on-vehicle→tripod” that may take place when the vehicle is stopped orthe engine is turned off, or mode shift “tripod→on-vehicle” that maytake place when the vehicle begins to run or the engine is started canbe achieved within a shortest period of time.

When hand-held image sensing is started from the power-ON timing of thecamera, since both counters M and N assured on a RAM are reset, i.e.,M=N=0, the hand-held image sensing mode is set in step S716.

Referring back to FIG. 41, in step S609 limiting characteristics thatlimit correction capability are selected from the panning characteristicdata 315 g ₂ on the basis of the vibration state determined in stepS608, and the limiting amount is calculated in accordance with theselected limiting characteristics. If the on-vehicle mode is determinedin step S609 a, the limiting amount is calculated by selecting panningcharacteristics I in step S609 c. If the tripod mode is determined instep S609 b, a limiting amount is calculated by selecting panningcharacteristics III in step S609 e. On the other hand, if the tripodmode is not determined in step S609 b, a normal hand-held mode isdetermined, and the limiting amount is calculated by selecting panningcharacteristics II in step S609 d. The process in step S609 is executedby the limiting process controller 315 g, and the determined limitingamount is reflected in step S604 in the next interrupt process. Notethat the limiting amount corresponds to the cutoff frequency that hasbeen explained in the band limiting process in step S604, and thelimiting characteristics based on the respective panning characteristicsare as shown in FIGS. 44A to 44C.

FIGS. 44A to 44C show characteristics of the limiting amount, i.e.,cutoff frequency, as a function of the correction amount. FIG. 44A showsthe limiting characteristics in the on-vehicle mode as panningcharacteristics I, FIG. 44B shows the limiting characteristics in thenormal hand-held mode as panning characteristics II, and FIG. 44C showsthe limiting characteristics in the tripod mode as panningcharacteristics III.

In FIGS. 44A to 44C, the abscissa plots the normalized correctionamount, and indicates the ratio of the correction amount required forcorrecting current vibration with respect to 100% when correction isdone by shifting to ½ the maximum shift limit (i.e., definitionof±maximum shift limits).

Note that the maximum shift limit is determined in advance as in theabove embodiment. The ordinate plots the band-limiting cutoff frequencyas a limiting amount parameter, and the cutoff frequency increases alongthe ordinate.

In this embodiment, the degree of limiting the correction amount is setnot based on a threshold value but by a function. For this reason, evenwhen panning is done by controlling the cutoff frequency, smoothswitching can be attained.

Also, in this embodiment, f_(C) ^(ref) represents the cutoff frequencywhen the normalized correction amount=0%. The cutoff frequency that mostsuppresses vibration with respect to the detected vibration frequency isset at f_(C) ^(ref). This will be explained in detail below using FIG.43.

FIGS. 43A and 43B show the frequency response characteristics from theangular velocity sensors 309 and 310 as vibration detection sensors tothe output of the vibration correction system. A curve 401 in FIG. 43Aindicates the gain characteristics, and a curve 402 in FIG. 43B indicatethe phase characteristics. Ideally, the anti-vibration frequency rangehas a flat gain and is free from any phase delay. However, in practice,the phase delays due to time delay from vibration detection until theoutput to the correction system, the response characteristics ofmechanical members, and the like, as the frequency is higher. If a phasedelay is caused by only time delay of a circuit system or processingsystem, a delay angle θ is given by:

θ(deg)=delay time (sec)*frequency (Hz)*360 (deg)  (8)

In practice, a filter and the like are designed to remove any phasedelay around 3 Hz to 5 Hz, so as to obtain the highest anti-vibrationeffect in the frequency band for normal hand-held image sensing.

Reference numeral 403 denotes a frequency band in which anti-vibrationcan be attained in this embodiment. For example, frequency f_(a)=1 Hz,f_(b)=4 Hz, f_(c)=20 Hz, and f_(d)=30 Hz (a frequency band 404 thatsuffers gain drop corresponds to an anti-vibration impossible band). Asshown in FIG. 43A, the gain is nearly flat, but the phase delay becomeslarger as the frequency becomes higher, as shown in FIG. 43B.

In the characteristics shown in FIGS. 43A and 43B, a phase angle θ_(b)is free from any phase delay at f_(b), the phase leads by θ_(a)-θ_(b) atf_(a), and the phase lags behind by θ_(b)-θ_(c) at f_(c).

With this phase shift, the anti-vibration effect lowers. In order toimprove suppression capability of anti-vibration for a frequency atwhich especially large phase delay occurs, the phase is advanced. As amethod of advancing phase, in this embodiment, the band-limitinghigh-pass filters 315 c ₁ and 315 c ₂ for band limitation are also used,and their cutoff frequencies are changed to advance the phase.

The cutoff frequencies f_(C) ^(ref) of the high-pass filters 315 c ₁ and315 c ₂, which compensate for the delay angle with respect to thefrequency at which the phase delay is produced are stored incorrespondence with the frequencies. During anti-vibration control, thecutoff frequency f_(C) ^(ref) to be phase-compensated is read out inaccordance with the detected vibration frequency and is controlled, thuspreventing deterioration of suppression capability of anti-vibration.

The cutoff frequency when the normalized correction amount=0% in FIGS.44A to 44C correspond to the cutoff frequency f_(C) ^(ref) thatcompensates for a phase delay shown in FIGS. 43A and 43B. If it isdetermined that vibration acting on the camera is caused by hand-heldimage sensing, the characteristics shown in FIG. 44B are selected instep S609 d, and the cutoff frequency is controlled in correspondencewith the normalized correction amount. In case of FIG. 44B, the maximumvalue of the cutoff frequency set is around 6 Hz, and f_(C) ^(ref) isaround 0.1 Hz. This is because ordinary shake frequency components fallwithin the range from about 3 Hz to 5 Hz, and there is no phase delay inthat frequency band.

Band-limiting characteristics 503 are set to change the cutoff frequencyas a quadratic function. With the characteristics 503, the cutofffrequency is controlled to increase sharply as the correction amountbecomes larger, and to be as low as possible when the correction amountis around zero so as to improve the anti-vibration effect.

If on-vehicle image sensing is determined, the characteristics shown inFIG. 44A are selected in step S609 c to control the cutoff frequency asa limiting amount. In case of on-vehicle image sensing, the vibrationfrequency becomes 20 Hz or higher, and the cutoff frequency thatcompensates for phase delay is set at f_(C) ^(ref).

In this embodiment, yaw and pitch characteristics 501 and 502 areillustrated to have identical f_(C) ^(ref), but they are merelyillustrated overlapping each other for easy comparison of characteristiccurves. In practice, since frequency components detected in units ofvibration directions independently change from time to time, f_(C)^(ref) in the yaw direction at a given time is often different from thatif vibration mainly agrees with the pitch direction, and the limitingcharacteristics in the pitch direction are set, as indicated by 505, soas to suppress small vibration components.Note that the cutoff frequencyfCref corresponding to the correction amhanged upon maneuvering, i.e.,limitation is smaller than that in the yaw direction.

With such characteristics, even in an image sensing situation in which avehicle bounces at joints of a road such as a bridge, image sensing canbe done with a stable frame free from the influence of vibration, andcamera work in the yaw direction can be prevented from being influencedby correction in the pitch direction.

If tripod image sensing is determined, the characteristics shown in FIG.44C are selected in step S6ount=0% is set at 0.1 Hz or less since thevibration frequency is in the neighborhood of zero in the tripod mode.The limiting characteristics in the yaw direction are set, as indicatedby 504. In the tripod mode since panning is done mainly in the horfvibration mainly agrees with the pitch direction, and the limitingcharacteristics in the pitch direction are set, as indicated by 505, soas to suppress small vibration components.

Note that the cutoff frequency f_(C) ^(ref) corresponding to thecorrection amount=0% is set at 0.1 Hz or less since the vibrationfrequency is in the neighborhood of zero in the tripod mode.

The limiting characteristics in the yaw direction are set, as indicatedby 504. In the tripod mode since panning is done mainly in thehorizontal direction, the characteristics are set to immediately obtaina maximum limiting amount upon panning if a still state is not detected.

Since the anti-vibration range of this embodiment has an upper limitnear f_(d)≈30 Hz, as described above using FIGS. 43A and 43B, f_(C)^(max)=50 Hz is set to have no gain with respect to the frequency bandbelow 30 Hz. In this embodiment, different limiting characteristics inthe tripod mode are set in correspondence with the vibration direction.However, panning in the tripod mode is done not only in the horizontaldirection but also in the vertical and oblique directions, although suchpanning is done not so frequently. In such image sensing situation, thecorrection system operates using the characteristics 505 in both the yawand pitch directions, and upon detecting vibration (NO is determined instep S707 in FIG. 42), the limiting characteristics in the detectedvibration direction are preferably switched to limiting characteristics504.

In this case, the predetermined value ω in step S708 in FIG. 42 can beset to assure a time longer than the expected camera work, i.e., panningtime. However, in such case, since the time required until the tripodmode is selected is also prolonged, different counters are preferablyused for counter N for determining the tripod mode in place of a commoncounter so as to determine shift to the tripod mode and cancel of thatmode, and predetermined values for mode shift determination arepreferably independently set in these counters, as shown in FIG. 42. Byselecting the limiting characteristics shown in FIG. 44C, a camera whichallows the user to make smooth camera work while removing the influenceof vibration at the location of the tripod can be provided.

Referring back to FIG. 41, the cutoff frequency determined based ondifferent characteristics in units of image sensing situations in stepS609 is set in the next band-limiting process to limit the angularvelocity signal. For example, when the calculated cutoff frequency ishigh, the correction effect lowers with respect to vibration having ashake frequency below the cutoff frequency. In step S610, the shifttarget value command calculated in step S606 is output to the adder 316,thus ending this process (step S611).

As described above, according to the eighth embodiment, the panningcharacteristics corresponding to different image sensing situations arepre-stored as a plurality of limiting characteristics, and optimallimiting characteristics are selected in accordance with the currentimage sensing situation, thus realizing smooth panning and obtaininghigh anti-vibration effect. When the image sensing situation isdiscriminated based on the detected vibration amplitude and frequency,optimal limiting characteristics can be automatically determined.

By providing a mode that controls vibrations in the pitch and yawdirections using different anti-vibration characteristics, even in anenvironment under specific vibration condition other than shake, optimalanti-vibration characteristics that can independently correct vibrationsin the pitch and yaw directions can be set. For example, even uponon-vehicle image sensing using a camera on a tripod, i.e., even whenvibration of the vehicle and the influence of the road are reflected inonly the pitch direction, a high-quality image sensing apparatus whichcan simultaneously satisfy high suppression capability of anti-vibrationin the pitch direction and smooth camera work in the yaw direction canbe provided. In addition, since optimal characteristics can beautomatically set for an image sensing situation in which theanti-vibration effect cannot be obtained in the prior art, an imagesensing apparatus which can broaden the image sensing region can berealized.

The eighth embodiment has been explained with reference to an opticalanti-vibration system using a shift lens. However, the present inventionis not limited to such specific arrangement. For example, correction maybe done by controlling the position of the extracted image using a fieldmemory, or electronic correction means that corrects vibration bycontrolling the extraction position using a large-scale or ultrahigh-resolution type CCD may be used. In this embodiment, angularvelocity sensors are used as vibration detection means. Alternatively, amotion vector detection or acceleration sensor may be used. In case ofan acceleration sensor, another integration process need only be addedinside or outside the anti-vibration control microcomputer.

In the description of FIG. 40, vibration angular displacementcalculation is implemented by software, but may be implemented byhardware.

Also, in the above description, the characteristics to be limited aredetermined as a function of the correction amount. Alternatively, as acharacteristic determination method, the characteristics may bedetermined by calculating equations or may be pre-stored as a data tablethat can obtain desired characteristics.

Furthermore, high-pass filters have been exemplified as limiting means.However, the present invention is not limited to such specific means aslong as a means can limit operation of the vibration correction means.For example, means for limiting the output from an integral filter bycontrolling the integral time constant of the integral filter may beused.

Modification of Eighth Embodiment . . . Second Modification

FIG. 45 shows the arrangement of a modification of the eighthembodiment, i.e., the second modification. In the eighth embodimentmentioned above, the limiting characteristics corresponding to thedetermined image sensing situation are automatically selected by thecamera. However, in some image sensing situations, optimalcharacteristics can hardly be automatically determined. In the secondmodification, to obtain natural camera work and high anti-vibrationeffect in such special image sensing situation, the user can set optimalanti-vibration characteristics corresponding to the image sensingsituation using a menu function provided to a television, video camera,home video recorder, or the like.

Note that the same reference numerals in FIG. 45 denote the same blocksas those in FIG. 40, and a detailed description thereof will be omitted.An image formed on the image sensing element 306 via the lens isphotoelectrically converted into an electrical signal, and the signal isamplified to optimal level by the amplifier 307. The amplified signal isinput to the camera signal processing circuit 308 and is converted intoa standard television signal. After that, the standard television signalis amplified to optimal level by an amplifier 601, and the amplifiedsignal is sent to a magnetic recording/playback apparatus 602. At thesame time, the amplified signal is sent to an LCD display circuit 603,thus displaying a sensed image on an LCD 604.

Note that the LCD 604 makes display to inform the photographer of animage sensing mode, image sensing state, alert, and the like. Theanti-vibration control microcomputer 315 controls a character generator607 to mix the output signal from the generator 607 in the LCD displaycircuit 603, thus superposing that display on the sensed image.

Reference numeral 606 denotes a menu function controller in themicrocomputer 315, which controls the character generator 607 inresponse to the operation state of a menu setting switch operated by thephotographer to display a menu window on the LCD 604. The menu windowdisplays a plurality of image sensing condition items (e.g., conditionssuch as white balance, remote-control reception, electronic zoom, andthe like), and setting conditions (e.g., ON and OFF for electronic zoom)for the respective items, and the photographer selects the item to beset and sets its condition (selection means).

To attain menu operation, the menu setting switch 605 includes a modeswitch for turning ON/OFF a menu function, a selection switch forselecting an item and its condition, and a determination switch fordetermining the selected contents. When the photographer operates theseswitches while observing the menu window, the menu controller controlsmenu window display in correspondence with key operation, thusrecognizing the set contents.

For example, assume that the image sensing condition items of the menufunction include item “select image sensing situation”, and the settingconditions of that item include choice of image sensing locations suchas “vehicle/ship/helicopter/footbridge”. When the photographer selectsand sets one of these conditions in accordance with his or her favor,the anti-vibration control microcomputer 315 selects anti-vibrationcharacteristics suitable for the image sensing situation, and executesanti-vibration control.

Also, the photographer may be allowed to select an effective frequencyrange from the menu to obtain the highest anti-vibration effect againstshake caused by the photographer himself or herself (the effectivefrequency normally ranges from 1 Hz to 3 Hz, and a frequency range from3 Hz to 5 Hz is set as the target vibration frequency to be suppressed,but the effective frequency may be shifted toward higher frequency, or amain target vibration frequency to be suppressed may be selected from aplurality of frequencies), and may also be allowed to select vibrationamplitude. For example, the gains of the amplifiers 311 and 312 thatdetermine the amplified angular velocity sensor outputs may be optimizedby selecting them from a plurality of gain candidates in correspondencewith shake of the photographer. In this way, by inputting conditionsselected by the photographer from the menu, an anti-vibration functionoptimal to the photographer can be provided.

Upon completion of menu setups, the vibration state discrimination unit315 g ₁ of the limiting process controller 315 g selects optimal panningcharacteriotoelectrically converted into an electrical Signal, and thesignal is amplified to optimal level by the amplifier 307. The amplifiedsignal is input to the camera signal processing circuit 308 and isconverted into a standard television signal. After thescribed above,according to the second modification, since the photographer can selectand instruct an image sensing situation, optimal anti-vibrationcharacteristics can be obtained even in an image sensing situation thatcan hardly be automatically discriminated.

When the effective frequency range is changed or the frequency band islimited in correspondence with the photographer, and the gain ofvibration amplitude is optimized in correspondence with different shakesin units of photographers, a shake correction function suitable for eachphotographer can be realized. In the second modification, thephotographer can select anti-vibration characteristics optimal to shakecaused by himself or herself. However, the object to be subjected toanti-vibration control is not limited to shake caused by thephotographer, and vibration at specific camera location may be removedby the same method.

According to the second modification, panning characteristicscorresponding to image sensing situations are pre-stored as a pluralityof limiting characteristics, and optimal limiting characteristics areselected in correspondence with the image sensing situation, thusobtaining smooth panning and high anti-vibration effect. Since the imagesensing situation can be automatically discriminated on the basis of thedetected vibration amplitude and frequency, optimal limitingcharacteristics can be automatically determined. When the photographerselects and instructs an image sensing situation, optimal anti-vibrationcharacteristics can be set even in an image sensing situation that canhardly be automatically discriminated. By providing a mode that controlsvibrations in the pitch and yaw directions using differentanti-vibration characteristics, even in an environment under specificvibration condition other than shake, optimal anti-vibrationcharacteristics that can independently correct vibrations in the pitchand yaw directions can be set. Hence, an image sensing apparatus thatcan implement natural camera work and high anti-vibration effect can beprovided.

Advantages of Eighth Embodiment and the Like

To restate, according to the eighth embodiment and second modification,panning characteristics corresponding to different image sensingsituations are pre-stored as a plurality of limiting characteristics,and optimal limiting characteristics are selected in correspondence withthe current image sensing situation, thus obtaining smooth panning andhigh anti-vibration effect.

By providing a mode that controls vibrations in the pitch and yawdirections using different anti-vibration characteristics, even in anenvironment under specific vibration condition other than shake, optimalanti-vibration characteristics that can independently correct vibrationsin the pitch and yaw directions can be set. As a result, optimalanti-vibration characteristics can be set even in a special imagesensing situation. For example, even upon on-vehicle image sensing usinga camera on a tripod, i.e., even when vibration of the vehicle and theinfluence of the road are reflected in only the pitch direction, ahigh-quality image sensing apparatus which can simultaneously satisfyhigh suppression capability of anti-vibration in the pitch direction andsmooth camera work in the yaw direction can be provided. In addition,since optimal characteristics can be automatically set for an imagesensing situation in which the anti-vibration effect cannot be obtainedin the prior art, an image sensing apparatus which can broaden the imagesensing region can be realized.

Since the photographer selects and instructs an image sensing situation,optimal anti-vibration characteristics can be set even in an imagesensing situation that can hardly be automatically discriminated. Hence,optimal anti-vibration characteristics can be set even in a specialimage sensing situation.

Especially, by changing the effective frequency range or limiting thefrequency band in correspondence with the photographer, and byoptimizing the gain of vibration amplitude in correspondence withdifferent shake amounts in units of photographers, a shake correctionfunction suitable for each photographer can be implemented.

The present invention can be further modified into various modificationsother than the embodiments as set forth above. For example, theembodiments alters the limiting characteristic to the image vibrationcorrection in accordance with image sensing condition. It may beproposed to modify the embodiments in which the characteristics areautomatically or manually altered in accordance with the condition.Further, the alteration of the limiting characteristics may be modifiedto be based on image-sensing condition other than those set forth in theabove embodiments, on conditions other than image sensing conditions.Furthermore, the alteration may be modified to be made voluntarily.

Alternatively, the maximum range in which the correction can be madeshould not be limited to the embodiments, thus, may be altered as theneed arises. The range may be set to a range which is not an actualmaximum correction range.

Alternatively, the present invention can be applied into modificationswhere the hardware arrangement of the above embodiments be substitutedwith software arrangements, or vice versa.

Alternatively, the present invention can be applied into modificationswhich are any combinations of the above described embodiments, or anycombinations of any elements of the embodiments that are required.

Alternatively, the present invention can be extended into one apparatus,an apparatus combined with another device, or any elements comprising adevice, that comprise of whole or partial elements of the appendedclaims or embodiments.

The present invention can be applied to a video cam-corder, video-stillimage camera, camera using silver-salt film, single-lens reflex camera,lens-shutter camera, observation camera, various types of cameras, anytypes of image sensing devices or optical devices other than cameras,and other types of devices. It can be further applied to a device whichis applied to the cameras, optical devices and the other types ofdevices, or to any component comprising the cameras, the optical devicesand the other types of devices.

As many apparently widely different embodiments of the present inventioncan be made without departing from the spirit and scope thereof, it isto be understood that the invention is not limited to the specificembodiments thereof except as defined in the appended claims.

What is claimed is:
 1. An apparatus comprising: a vibration correctiondevice for correcting a vibration of an image; and a limiting device forcalculating a ratio of an amount of vibration correction to be effectedby said vibration correction device in accordance with an image shakecondition, to a maximum value of vibration correction by said vibrationcorrection device, and limiting an operation of said vibrationcorrection device in correspondence with a calculation result.
 2. Theapparatus according to claim 1, wherein said vibration correction devicecorrects vibration of the image by signal processing.
 3. The apparatusaccording to claim 1, wherein said vibration correction device opticallycorrects vibration of the image.
 4. The apparatus according to claim 1,further comprising vibration detection device for detecting vibration,and wherein said vibration correction device operates in accordance withan output from said vibration detection device, and said limiting devicelimits the operation of said vibration correction device in accordancewith the output from said vibration detection device.
 5. The apparatusaccording to claim 1, wherein said limiting device determines a value ofa limiting amount in proportion to the n-th power (n is an integer notless than 1) of a value of the correction amount determined by saidvibration correction device.
 6. The apparatus according to claim 1,wherein said apparatus comprises an image sensing apparatus.
 7. Theapparatus according to claim 1, wherein said apparatus comprises acamera.
 8. The apparatus according to claim 1, wherein said apparatuscomprises an optical apparatus.
 9. The apparatus according to claim 1,wherein said limiting device is a variable value device.
 10. Theapparatus according to claim 1, wherein said limiting device is adjustedto decrease anti-vibration correction during panning and adjusted toincrease anti-vibration during normal image sensing.
 11. The apparatusaccording to claim 1, wherein said limiting device changes in value tocorrespond with the changes in a focal length and vibration correctionamounts.
 12. A vibration correction method comprising the steps of:calculating a ratio of an amount of vibration correction to be effectedby a vibration correction device in accordance with an image shakecondition, to a maximum value of vibration correction by said vibrationcorrection device; and limiting an operation of said vibrationcorrection device in correspondence with a calculation result in saidcalculating step.
 13. The method according to claim 12, wherein saidvibration correction device corrects vibration of the image by signalprocessing.
 14. The method according to claim 12, wherein said vibrationcorrection device optically corrects vibration of the image.
 15. Themethod according to claim 12, wherein said vibration correction deviceoperates in accordance with an output from a vibration detecting device,and in said limiting step, the operation of said vibration correctiondevice is limited in accordance with the output from said vibrationdetection device.
 16. The method according to claim 12, wherein in saidlimiting step, a value of a limiting amount is determined in proportionto the n-th power (n is an integer not less than 1) of a value of thecorrection amount determined by said vibration correction device.